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INTRODUCTION 


An  Occupational  Analysis  Seminar,  sponsored  by  the 
Establishments  Division,  Department  of  Defence  Central 
Office,  was  conducted  in  Canberra  from  17-20  September  1979* 

Guest  speaker  was  Dr  Raymond  E.  Christal  of  the 
United  States  Air  Force  Human  Resources  Laboratory  in  Texas. 
Dr  Christal  was  Technical  Director  of  the  USAF  Occupation  and 
Manpower  Research  Program, 

Members  of  the  Australian  Defence  Force  attended  the 
Seminar  in  company  with  representatives  from  the  State 
Electricity  Commission  of  Victoria,  the  Public  Service  Board, 
the  Department  of  Social  Security  and  the  Department  of 
Employment  and  Youth  Affairs. 
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OPENING  ADDRESS  BY 
MAJOR  GENERAL  J.C.  HUGHES,  DSO  MC 
CONTROLLER  OP  ESTABLISHMENTS 


Ladies  and  Gentlemen, 

It  gives  me  great  pleasure  to  welcome  you  all  to  this 
Occupational  Analysis  Seminar,  especially  those  from: 

the  United  States  Air  Force 

the  New  Zealand  Defence  Liaison  Staff 

the  Burwood  State  College 

the  State  Electricity  Commission  of  Victoria 

the  Committee  of  Reference  for  Defence  Force  Pay 

the  Public  Service  Board 

the  Department  of  Social  Security 

the  Department  of  Employment  and  Youth  Affairs 

the  ACT  Curriculum  Development  Centre,  and 

the  Bruce  College  of  Technical  and  Further  Education 

Australia's  formal  interest  in  Occupational  Analysis  commenced 
in  August  1969  when  the  Principle  Administrative  Officers'  Committee 
(Personnel)  endorsed  a  paper  tabled  by  the  Air  Member  for  Personnel 
which  recommended  the  introduction  of  Job  Analysis  procedures  into 
the  Australian  Defence  Force  and  the  establishment  of  Job  Analysis 
Sections  in  each  of  the  three  Services  and  in  the  Department  of 
Defence . 


My  interest  in  Job  Analysis,  or  Occupational  Analysis  as  we 
now  call  it,  stems  from  the  fact  that  in  early  1975  Defence  Central 
Office  responsibilities  for  Job  Evaluation  Studies  and  Occupational 
Analysis  were  vested  in  my  Division  and  a  Special  Section  was  established 
to  advise  the  Service  Offices  on  the  implementation  of  Occupational 
Analysis  techniques  and  to  co-ordinate,  as  required,  Occupational 
Analysis  activities  which  have  Joint-Service  implications. 

The  Aims  of  the  Seminar  are,  first,  to  examine  the  further 
application  of  Occupational  Analysis  techniques  to  the  personnel 
requirements  of  the  Australian  Defence  Force;  and  second,  to  familiarize 
Senior  Officers  with  the  potential  value  of  the  CODAP  system  as  a 
management  tool. 
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In  these  days  of  financial  constraints  and  expensive  manpower 
costs,  there  is  a  need  to  develop  and  refine  techniques  for  efficient 
manpower  management.  Managers  are  becoming  more  aware  of  the  need 
for  techniques  for  gathering  reliable  information  and  rapid  updating. 
This  implies  the  existence  of  a  comprehensive  Occupational  Data 
Bank.  Also  there  is  need  for  a  more  systematic  approach  to  problem 
solving  and  for  computer  assistance  in  the  areas  of  complex  problem 
solving,  such  as  in  contingency  planning. 

This  Seminar  is  the  culmination,  indeed  the  high  point,  of  a 
long  period  of  assistance  from  the  United  States.  I  refer 
specifically  to  our  guest  speaker,  Dr  Ray  Christal  and  to  the  United 
States  Air  Force  exchange  officers.  Having  adopted  procedures  based 
on  the  American  experience  and  having  had  some  experience  with 
practical  applications,  we  now  seek  to  "firm-up"  on  our  future  plans 
and  to  resolve  outstanding  problems. 

I  hope  that,  as  a  result  of  this  Seminar,  Senior  Officers  will 
gain  an  understanding  of  the  fundamentals  of  Occupational  Analysis, 
the  practice  of  which,  at  present,  is  restricted  mainly  to  more 
Junior  Officers.  I  hope  also  that  the  practitioners  will  advance 
their  knowledge  of  the  techniques  involved  and  gain  the  level  of 
expertise  which  will  enable  them  to  further  develop  procedures.  As 
a  result,  each  of  the  Services  and  the  Department  of  Defence  as  a  whole 
will  benefit. 

In  i  Is  explanations  of  the  word  "seminar",  the  Oxford  Dictionary 
includes  "conference  of  specialists"  and  "short  intensive  Course  of 
Study".  This  Seminar  is  inclined  more  to  the  former.  Ample  time 
is  allowed  for  discussion  of  each  topic  and  maximum  participation  by 
all  is  our  hope.  Let  us  hear  your  views. 
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HISTORY  OF  JOB  SURVEY  AND 
CODAP  ANALYSIS  SYSTEM 


By  R.B.  Christal,  BS,  MS,  PhD 
Technical  Director 

Occupation  and  Manpower  Research  Division 
Air  Force  Human  Resources  Laboratory, 
United  States  Air  Force 


INTRODUCTION 


1 .  At  the  outset,  I  have  to  say  that  my  experience  has 
been  primarily  with  the  United  States  Air  Force  (USAF).  The 
Air  Force  has  a  simpler  classification  structure  than  the  other 
services.  Most  of  our  people  are  at  large  bases;  and  we  don't 
have  to  worry  about  the  combat  aims  problem  where  what  you  do 
in  peacetime  may  not  be  what  you  do  in  wartime.  For  these 
reasons,  I  believe  that  in  many  ways  the  Air  Force  is  easier 

to  manage  than  the  other  services. 

2.  So  that  you  will  be  familiar  with  the  terms  I'm  using, 
I'll  describe  to  you  the  classification  structures  in  the  USAF. 
The  first  tables  are  for  enlisted  airmen.  (Tables  1  &  2). 

There  are  48  career  fields.  In  the  medical  career  field,  there 
are  28  career  ladders.  Within  the  48  career  fields,  there  are 
334  career  ladders.  In  a  career  ladder,  a  person  starts  at 
entry  level  and  goes  from  apprentice  to  journeyman  to  first- 
level  supervisor,  and  finally,  to  general  manager.  The  skill 
levels  associated  with  those  terms  are  numbered  1,  3>  5>  7,  9, 
and  0.  There  is  a  grade  spread  associated  with  each  skill 
level,  and  the  pay  level  is  directly  related  with  the  number 

of  stripes  a  man  wears.  \ 

_ _ 


NO 

CATEGORY 

EXAMPLE 

48 

Career  Fields 

Medical  (28  Ladders) 

334 

Career  Ladders 

Pharmacy 

1000+ 

Specialties 

.Pharmacist  Specialist 
(Journeyman) 

Table  1 .  USAF  Classification  Structure  (Enlisted) 


GENERAL 

LEVEL 

OFFICIAL 

GRADE 

TITLE 

TITLE 

RANGE 

Entry  Level 

1 

Airman  Basic 

El 

Apprentice 

3 

Semi-skilled 

E1-E3 

Journeyman 

5 

Skilled 

E3-E5 

1 st  Line 
Supervisor 

7 

Advanced 

E5-E7 

Superintendent 

9 

Super int  endent 

E7-E8 

General  Manager 

0 

Chief  Enlisted 
Manager 

E9 

Table  2.  Skill-Level-Grade  Relationships 
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3.  The  chart  for  the  officers'  classification  structure 

shows  22  career  areas  made  up  from  46  utilization  fields. 
(Table  3).  A  pilot  is  in  a  utilization  field,  but  a  pilot, 
fighter-interceptor,  FI 06,  is  in  a  specialty.  The  officers' 
grade  levels  are  divided  loosely  into  three  groups:  company 
grade,  staff,  and  commanders  and  directors  (who  have  most  of 
the  higher  level  managerial  positions). 


NO 

CATEGORY 

EXAMPLE 

22 

Career  Areas 

Scientific  and  Engin¬ 
eering 

46 

Utilization 

fields 

Pilot 

598 

Specialties 

Pilot,  Fighter- 
Interceptor,  FI  06 

Table  3.  USAF  Classification  Structure 
(Officers) 


JOB  SURVEY  TECHNOLOGY 

4.  Turning  to  the  history  of  job  surveying  and  CODAP 
analysis  technology,  our  research  began  in  1957.  Headquarters 
USAF  wanted  quantitative  information  about  what  people  were 
doing  on  the  job.  At  first,  we  were  reluctant  to  accept  that 
responsibility.  To  us,  job  analysis  was  more  an  area  for 
practitioners  than  for  researchers.  Job  analysts  conducted 
interviews  and  time  and  motion  studies  —  this  was  not  our 
forte.  We  doubted  whether  research  in  this  area  would  have  any 
payoff.  But  now  I  must  admit  that  managers  often  have  more 
insight  than  researchers.  The  domain  was  researchable ,  and 
the  technology  coming  from  this  research  has  had  many  payoffs. 

5.  In  1957,  the  Air  Force  already  had  an  occupational 
survey  technique.  At  the  local  level,  individuals  wrote  job 
descriptions  under  supervision.  These  job  descriptions  would 
come  In  to  a  central  location  from  all  over  the  world.  They 
came  in  great  quantities  and  were  usually  handwritten.  The 
problem  was  that  no  one  knew  what  to  do  with  them.  One  could 
get  a  general  idea  of  what  was  happening  in  a  specialty,  but  he 
couldn't  count  how  many  people  performed  a  particular  work  task. 
People  wrote  descriptions  at  different  levels  of  specificity; 
some  wrote  broad  task  statements,  and  some  wrote  narrow  task 
statements.  They  were  very  difficult  to  collate.  The  info¬ 
rmation  was  not  very  useful  for  trainers;  and  it  didn't  help 
the  classification  people  to  determine  the  occupational  cate¬ 
gories.  The  descriptions  were  most  often  used  to  defend  actions 
taken.  For  example,  someone  would  ask  why  an  occupational 
category  had  been  broken  into  two  categories.  And  the  answer 
was:  "we  analysed  this  stack  of  job  descriptions!"  But,  to 
give  the  classification  structure  group  credit,  it  recognized 
that  there  was  a  problem  and  requested  information  so  that  it 
could  do  a  better  job  of  resolving  management  problems.  It 
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was  at  their  request  that  we  established  our  research  program. 

6.  The  problem  looked  formidable.  There  were  700,000 

Air  Force  personnel  performing  tens  of  thousands  of  different 
types  of  work  tasks  at  hundreds  of  locations  throughout  the 
world.  How  could  we  find  out  what  all  of  those  people  are 

doing?  What  a  person  actually  did  was  dictated  by  his  first- 

line  supervisor.  For  each  job,  the  only  descriptive  information 
we  had  was  a  five-digit  number  indicating  the  occupation  and 
skill-level.  Clearly,  we  had  a  communications  problem.  There 
just  simply  wasn't  enough  information  coming  back  from  the  field 
to  tell  the  trainers  what  kinds  of  training  individuals  needed 
in  order  to  perform  their  jobs  adequately.  Well,  we  studied 
different  systems  for  collecting  occupational  data  and  finally 
concluded  that  the  task  inventory  approach  was  the  best. 
Fortunately,  computers  were  coming  on-line,  and  we  thought  we 
could  handle  very  detailed  information  provided  we  could  collect 
it . 


7.  The  task  inventory  which  we  developed  for  occupational 
surveys  is  a  very  simple  instrument .  It  contains  a  Background 
Information  section  in  which  we  get  general  information  about  the 
worker  and  his  job.  Also,  in  this  section,  we  collect  attitu- 
dinal  information.  We  ask  incumbents  whether  their  job  is 
interesting.  The  background  information  is  very  important  be¬ 
cause  our  ability  to  address  management  questions  later  on  is 

a  function  of  whether  we  have  information  to  identify  subsets  of 
people  in  which  we  are  interested.  I  would  like  to  stress  the 
importance  of  looking  at  management ' s  needs  before  sending  the 
inventory  into  the  field.  The  second  part  of  the  inventory  is 
a  list  of  all  the  tasks  performed  by  anyone  in  an  occupational 
category.  It  takes  a  long  time  to  develop  a  list  but,  once 
you  have  it,  it  becomes  a  very  useful  instrument.  Every 
person  in  an  occupation  should  be  able  to  define  what  he  does 
in  terms  oi  a  subset  of  tasks  in  that  basic  list. 

8.  How  do  we  get  information  directly  from  workers  in  the 
field?  Well,  first,  the  incumbent  fills  out  the  Background 
Information  section:  then  he  reads  all  the  tasks  in  the  inven¬ 
tory  and  checks  (  the  ones  he  performs.  Also,  he  writes  in 
any  tasks  that  he  performs  that  are  not  in  the  basic  list. 

Next,  he  reads  through  the  list  again  and,  using  a  simple  scale, 
indicates  the  relative  amount  of  time  spent  on  each  task.  Now 
you  can  see  why  we  selected  the  inventory  approach.  You  get 
quantitative  data,  and  you  can  count  how  many  people  do  a  task 
at  each  location.  It's  a  relatively  fast  procedure  in  that  you 
get  information  from  people  scattered  across  the  nation  in  a 
matter  of  weeks.  On  the  other  hand,  if  you  went  into  the  field 
to  interview  people,  it  would  take  a  lot  longer  to  get  a  substan¬ 
tial  sample.  The  task  inventory  data  is  analysed  by  computer; 
and  you  can  generate  all  sorts  of  reports  to  service  management 
needs.  Finally,  the  results  are  subject  to  research  and 
verification. 


CODAP  PROGRAMS 


9.  Occupation  surveys  using  task  inventories  is  only  half 
of  the  technology.  The  other  half  is  called  CODAP  (Comprehen¬ 
sive  Occupational  Data  Analysis  Programs).  CGDAP  has  five 
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classes  of  programs: 

Job  Description  Programs 
Job-Typing  Programs 
He search  Programs 
Task  factor  Programs 
Interactive  Programs 

10.  Job  Description  Programs  describe  what  people  are  doing. 
You  can  get  a  description  of  one  person's  job.  It  will  list 
every  task  he  does  and  the  tine  he  spends  on  each  task.  If  you 
have  a  group  of  people  at  some  location  who  r  eport  that  their 
talents  are  not  being  utilized,  you  can  print  out  a  description 
for  each  person  in  the  group,  one  at  a  time.  Usually  we 
compute  group  job  descriptions.  For  example,  you  might  wish  to 
have  a  job  description  for  women  working  at  a  particular  job 
location,  or  for  sailors  on  a  particular  class  of  ship.  The 
computer  will  produce  a  job  description  for  any  group  that  can 

be  defined  in  terms  of  information  collected  in  the  background 
section  of  the  task  inventory.  Managers  may  want  to  know  if 
there  are  differences  in  work  being  performed  by  two  groups. 

For  example,  is  there  any  difference  in  the  work  being  performed 
by  incumbents  in  adjacent  grade  levels?  You  can  ask  for  a 
difference  description  for  the  people  performing  their  activit¬ 
ies  in  Grade  E4  versus  Grade  E5*  You  can  have  difference  des¬ 
criptions  for  men  versus  women,  for  individuals  on  one  ship 
versus  another,  and  so  on.  You  can  also  request  a  group 
summary  which  lists  the  tasks  in  the  inventory  by  rows,  and 
any  other  variable  by  columns.  Column  headings  may  be  for 
any  breakout,  such  as  for  class  of  ship,  aircraft  type,  or 
time-in-service.  If  the  columns  are  constructed  in  terms  of 
months  of  service  categories,  you  can  read  across  the  task  rows 
and  find  out  when  tasks  tend  to  come  into  play.  This  is  a  useful 
output  when  you  want  to  find  out  when  training  should  be  given 
to  be  timely. 

11.  Next,  we  have  job-type  programs  which  are  very  impor¬ 
tant  for  decisions  concerning  the  classification  structure 
problems.  Job-type  programs  compare  the  work  being  performed 
by  all  individuals  in  a  sample.  They  pull  together  people 
doing  the  same  tasks,  and  define  the  types  of  jobs  which  exist 
in  an  occupation.  For  example,  job-type  analyses  revealed 
there  are  over  20  distinct  types  of  jobs  in  the  Air  Force's 
Materiel  Facilities  career  ladder,  even  though  we  act  as 
though  it  is  a  single  occupation.  This  sort  of  information 
is  very  important  when  you  are  considering  changes  needed  in 
your  occupational  categories. 

12.  Research  programs  are  used  to  address  many  management 
questions.  Take  the  variable  generation  program  as  an  example. 

If  you  want  to  know  how  much  time  people  spend  on  paperwork, 
you  can  have  supervisors  check  all  the  tasks  in  an  inventory 
which  are  associated  with  paperwork.  Then  you  can  calculate 
the  amount  of  work-time  each  person  spends  on  that  set  of  tasks 
or  produce  information  concerning  time  spent  on  paperwork  by 
various  classes  of  people.  If  you  wish,  you  can  produce  a 
two-way  distribution  between  job  satisfaction  and  time-in¬ 
service  and  find  out  whether  job  satisfaction  tends  to  deter¬ 
iorate  at  some  point  in  time.  Also,  there  is  a  reliability 


program  which  enables  you  to  determine  whether  the  raters  agree 
with  each  other  when  they  provide  judgements  on  such  things  as 
the  difficulty  levels  of  tasks  in  their  occupation.  There  is 
a  correlation/regression  routine  in  the  GODAP  system  which  is 
useful  for  developing  models.  For  example,  you  are  trying 
now  to  get  an  equation  to  determine  the  worth  of  jobs  for 
pay  purposes.  The  CODAP  regression  program  should  be  very 
useful  in  helping  you  to  accomplish  this. 

Task  factor  programs  form  the  fourth  class  of  programs. 
«:.ut  are  the  consequences  of  various  tasks  not  being  performed 
:uately?  How  difficult  are  particular  tasks?  If  you  want 
rm.ition  about  the  nature  of  a  task,  you  can  collect  super- 
ratings  and  use  CODAP  programs  to  analyse  the  information 
'  :■  v  u. 

i .  The  fifth  class  of  CODAP  programs  provides  the  means 

r  interacting  job  information  with  task  information.  For 
exampie,  there  are  programs  for  computing  the  difficulty  of 
*  r.<  performed  or  the  average  level  of  hazard  faced  by  each 
individual  surveyed.  It  would  be  impossible  to  indicate  the 
full  capabilities  of  the  CODAP  system  in  the  tine  available 
today.  For  this  reason,  I  have  included  on  Annex  B  a  set  of 
abbreviated  CODAP  program  descriptions  prepared  by  Michael 
Thew  and  Johnny  Weissmuller,  lead  programmers  for  the  CODAP 
system  in  the  Air  Force. 


1UESTI0NS  ABOUT  THE 
OB  SURVEY  APPROACH 


15.  Should  one  begin  with  surveys  of  officers  or  with 
surveys  of  enlisted  personnel?  We  decided  to  begin  with 
enlisted  personnel,  for  a  number  of  reasons.  First,  there's 
a  lot  more  information  available  to  construct  inventories  for 
enlisted  personnel,  including  training  standards,  training 
course  outlines  and  so  on.  Secondly,  there  are  a  lot  more 
training  courses  for  enlisted  personnel,  so  there  is  greater 
possible  money  savings.  Thirdly,  the  cost  of  administration 
per  person  is  less  because  the  sample  sizes  are  larger. 

Fourthly,  the  tasks  that  enlisted  people  do  are  more  visible; 
you  can  observe  and  verify  what  they  claim  0  be  doing. 

Finally,  because  of  protocol  problems,  it's  sometimes  difficult 
to  interview  high  level  officers  required  for  the  construction 
of  officer  inventories.  Despite  the  special  problems  with 
officers,  I  should  mention  that  we  have  collected  good  officer 
data  for  specialties  where  the  work  is  content-oriented. 

However,  the  cost  of  inventory  construction  is  at  least  three 
times  higher  for  officer  inventories  than  for  enlisted  inven¬ 
tories,  and  the  payoffs  of  survey  results  have  been  smaller. 

16.  Should  you  collect  information  from  the  worker,  or  from 
his  supervisor?  We  decided  on  the  worker.  The  first  problem 
was  that  we  wanted  to  survey  everyone,  including  the  supervisors, 
and  the  supervisors  with  many  subordinates  would  have  to  work 
very  hard  to  provide  detailed  information  on  all  subordinates. 
Secondly,  we  believed  that  supervisors  knew  a  lot  about  the 


general  nature  of  the  work  performed  by  their  subordinates, 
but  they  do  not  know  how  work-time  is  distributed  across  tasks. 
Thirdly,  there  was  the  possibility  of  some  gaming  if  the 
information  was  provided  by  supervisors  only.  This  may  or 
may  not  have  been  a  valid  concern. 

17.  What  level  of  task  specificity  should  be  used?  We 
have  concluded  that  you  should  not  hold  to  a  single  level  of 
specificity.  You  can  write  relatively  broad  task  statements 
for  supervisors  because  they  tend  to  supervise  whole  shops. 

With  the  worker  himself,  you  have  to  be  very  specific. 
Specificity  is  related  to  the  length  of  inventory.  We  don't 
like  for  a  man  to  spend  more  than  two  hours  in  filling  one  out. 
However,  it's  amazing  how  many  tasks  you  can  get  information 
on  in  two  hours  if  you  organize  them  under  duties.  People 
can  then  skip  large  numbers  of  irrelevant  duties  and  check  only 
those  tasks  under  duties  which  they  perform.  Our  experience 
has  been  that  our  inventories  have  become  longer  over  the  years. 
During  the  early  time  period,  some  trainers  complained  that  some 
tasks  were  not  specific  enough  for  them  to  establish  training 
objectives.  If  you  write  tasks  which  are  too  broad,  you  get 
little  from  the  job-type  analyses.  With  very  broad  statements, 
there  is  also  the  danger  that  two  people  checking  a  task  don't 
do  the  same  thing.  For  these  reasons,  we  have  tended  to  make  ■ 
our  tasks  more  and  more  specific  during  recent  years. 

18.  What  kind  of  rating  scale  should  be  used?  We  use  a 
time-spent  scale  in  preference  to  a  frequency  scale.  If  you 
use  frequency,  all  you  can  do  with  it  is  to  report  frequency 
distributions.  Our  "percent  time"  estimates  have  proved  very 
useful.  Time  percentages  have  a  clearly  defined  range  and  a 
base  value  of  "0".  For  the  individual  case,  the  values  sum 
to  100$.  In  a  like  manner,  the  average  values  for  any  group 
of  workers  also  sum  to  100$.  Percent  time  values  provide  a 
convenient  method  for  computing  the  overlap  of  two  individuals 
or  groups  on  specified  subsets  of  tasks.  All  in  all,  we  have 
been  very  satisfied  with  our  decision  to  use  percent  time  spent 
on  tasks  as  our  basic  job  information.  These  values  may  not 
be  accurate  enough  for  some  applications,  such  as  precise  cost 
analyses;  but  they  are  accurate  enough  for  job- type  analyses, 
job  evaluations,  and  job  descriptions. 

19*  How  broad  should  be  the  surveys?  It's  advisable  to 

cover  a  whole  occupational  category  from  entry  level  to  super¬ 
intendent  in  a  single  inventory.  Then  you  can  find  out  the 
extent  to  which  people  are  performing  tasks  appropriate  to  their 
grade  and  skill  level. 

20.  Should  a  common  set  of  verbs  be  used?  Some  people 
say  that  when  you  write  task  statements  you  should  use  a  set 
of  carefully-defined  standard  verbs.  We  don't  do  that.  We 
use  the  verbs  that  the  worker  uses  in  his  particular  occupation. 

21 .  How  large  should  be  the  survey  samples?  If  you  only 
want  one  job  description  of  what  people  are  doing  in  an  occupa¬ 
tion,  then  several  hundred  cases  will  give  stable  information. 
But  you  may  want  to  know  what  work  is  being  done  by  females, 
unmarried,  who  are  on  their  first  assignment.  If  you  have  only 
400  cases  and  you  pull  out  the  female  subset,  you  may  only  find 
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one  or  two  cases.  If  you  want  a  data  bank  which  is  useful  in 
addressing  numerous  management  problems,  then  you  must  have  large 
samples.  The  additional  cost  is  not  very  significant.  The 
main  cost  is  in  the  construction  of  the  inventory. 

22.  What  is  the  cheapest  way  to  construct  inventories? 

Some  of  the  United  States  services  construct  inventories  by 
interviews  only.  We  found  that  after  you  have  interviewed 
six  to  twelve  people,  you  get  diminishing  returns  in  terms  of 
identifying  new  tasks.  We  start  with  written  sources  of  in¬ 
formation,  prepare  an  initial  inventory,  and  then  interview  a 
few  people  in  the  occupation.  We  add  those  tasks  that  are 
suggested  and  then  mail  the  revised  list  to  supervisors  in  all 
kinds  of  organizations  in  many  locations.  These  field  super¬ 
visors  add  tasks  which  are  not  in  the  list  but  which  are  being 
performed  in  their  areas  of  responsibility.  We  find  this  the 
most  economical  method  for  developing  a  comprehensive  task  test. 

23.  How  should  you  collect  occupational  data?  Some  serv¬ 
ices  insist  on  group  administration  under  controlled  conditions. 
Incumbents  are  given  thorough  instructions  before  filling  in 
the  inventory.  We  started  in  this  way,  but  found  that  some 
booklets  coming  from  the  field  had  grease  fingerprints  on  them, 
as  though  not  everyone  was  going  to  the  testing  room.  When  we 
researched  this  problem,  we  found  that  some  people  filled  the 
inventory  out  on  the  job,  some  in  testing  rooms,  and  some  at 
home .  We  divided  the  samples  into  those  who  were  administered 
the  inventory  under  controlled  conditions  and  those  who  weren't. 
We  couldn't  find  any  difference  in  the  information  that  we  got 
back  from  these  groups.  Although  the  regulation  about  cont¬ 
rolled  administration  still  exists,  we  don't  give  it  too  much 
attention.  If  people  are  filling  out  inventories  on  their  own 
time,  then  it's  cheaper  for  the  organization.  The  important 
thing  is  to  have  a  control  so  that  the  inventories  are  returned 
to  a  central  point  for  processing.  We  don't  like  to  see 
inventories  going  through  the  hands  of  supervisors,  especially 
if  you  have  attitudinal  questions  in  them. 

24.  What  formats  should  we  have  for  the  outputs?  If  you 
don't  make  them  simple,  people  won't  use  the  reports.  You  must 
have  your  task  statements  and  your  information  side  by  side. 

For  trainers,  you  only  give  them  what  is  relevant  to  training 
decisions. 


25.  Who  should  construct  inventories?  Inventory  construct¬ 
ion  is  an  art,  and  that  is  the  weakest  part  of  the  whole 
technology.  It  takes  a  lot  of  training  for  a  person  to  become 
proficient.  I  don't  think  people  in  teaching  jobs  should 
construct  inventories.  They  tend  to  emphasize  only  those  tasks 
which  are  relevant  to  training.  You  must  use  someone  who  can 

go  to  all  sorts  of  managers  and  find  out  the  nature  of  their 
problems.  I  believe  that  a  full-time  professional  staff  does 
this  job  best.  On  the  other  tend,  if  the  trainers  don't 
participate  in  the  inventory  construction  and  are  not  ego  inv¬ 
olved,  they  won't  trust  the  information  and  they  won't  use  it. 
Therefore,  they  must  be  used  in  the  review  stage. 

26.  Should  one  collect  names  of  incumbents?  We  always  get 
people's  names.  If  you  don't  get  names  and  identification, 
you  can't  verify  the  accuracy  of  information.  Secondly,  if  you 
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want  to  find  out  the  impact  of  job  changes  on  job  satisfaction, 
you  have  to  identify  a  group  of  people  at  different  points  in 
time.  Unlike  personality  inventory  data,  task  data  are 
objectively  verifiable.  It's  hard  for  an  incumbent  to  say 
he  does  a  task  when  everyone  around  him  knows  that  he  doesn't. 

One  reason  that  we  get  good  quality  information  is  because 
people  sign  their  names.  On  the  question  of  privacy,  the  Air 
Force  believes  that  management  has  a  right  to  know  what  the 
workers  are  doing.  With  attitudinal  questions,  the  worker  gives 
information  voluntarily.  We  have  taken  names  from  hundreds 
of  thousands  of  cases  and  I  can't  remember  hearing  of  an 
objection  being  lodged. 

27.  How  do  we  know  that  the  information  is  good?  First,  we 

had  a  contract  with  the  University  of  Texas  which  showed  that 
you  can  trust  people  to  write  in  tasks  that  they  are  doing  but 
which  are  not  in  the  basic  list.  Then,  we  introduced  veracity 
checks  in  which  we  placed  a  task  in  an  inventory  in  two  differ¬ 
ent  locations  to  see  if  the  person  gives  the  same  information 
twice.  This  tells  you  if  he  is  reading  the  tasks.  We  check 
to  see  whether  apprentices  are  claiming  they  do  superintendent's 
tasks.  We  have  test-retest  studies  in  which  we  had  incumbents 
fill  out  inventories  several  weeks  apart  to  see  if  they  provid¬ 
ed  consistent  information.  We  found  differences  which  worried 
us  for  a  while.  Then  we  found  that  many  jobs  change  fairly 
quickly.  What  an  organization  does  remains  fairly  stable,  but 
the  individuals'  jobs  change  over  time.  We  also  compared 
people's  time  cards  with  what  they  put  in  an  inventory.  We 
telephoned  people  in  the  field  and  asked  them  what  they  meant 
when  they  said  they  performed  certain  tasks.  Sometimes  we 
found  that  we  didn't  have  the  right  tasks  in  the  inventory  to 
capture  the  true  nature  of  a  job.  For  example,  we  found  one 
supply  group  that  spent  more  time  obtaining  signatures  than 
issuing  items.  Then  we  found  that  a  new  type  of  job  had  come 
into  play  where  expediters  were  delivering  goods  from  the 
warehouse  to  the  flight  line.  They  obtained  signatures  for  the 
goods,  but  we  didn't  have  a  task  in  the  inventory  concerning 
driving  parts  from  a  warehouse  to  a  flight  line.  The  real  test 
of  the  quality  of  survey  data  came  when  people  started  to 
challenge  results.  For  example,  in  a  survey  of  the  accounting 
and  finance  area,  the  occupational  survey  uncovered  only  14 
military  personnel  who  claimed  to  be  performing  civilian  pay 
functions.  Yet,  fully  25  percent  of  the  content  of  the  military 
Career  Development  course  was  devoted  to  such  tasks.  The 
Comptroller  felt  the  survey  data  couldn't  be  correct,  so  he 
canvassed  every  Accounting  and  Finance  Office  to  locate  all 
military  personnel  performing  tasks  related  to  civilian  pay. 

He  was  able  to  locate  only  17  individuals.  We  projected  from 
the  occupational  survey  sample  to  the  complete  population,  and 
the  results  predicted  he  should  have  found  17.  This  one  study 
did  much  to  convince  HQ  USAF  that  the  survey  procedure  yields 
thrustworthy  results.  Considerable  evidence  has  accumulated 
over  time  which  gives  us  confidence  in  the  data.  Many  changes 
have  been  made  on  training  courses  and  in  the  classification 
structures  based  upon  survey  data,  and  these  changes  have 
survived.  The  very  fact  that  the  technology  is  still  generally 
accepted  after  20  years  is  evidence  of  its  power  and  validity. 

We  have  many  reports  from  agencies  outside  the  Air  Force  who  are 
convinced  that  they  get  good  information  using  occupational  survey 
and  CODAP  analysis  procedures.  Chart  5  in  Annex  C  provides  a 


partial  list  of  the  many  agencies  and  organizations  using  this 
technology.  The  Marine  Corps  modified  their  complete  occup¬ 
ational  structure  using  the  data  from  the  CO DAP  system  and  re¬ 
ceived  a  Presidential  Management  Award  for  this  effort.  Apart 
from  the  military,  there  is  a  consortium  of  States  within  the 
United  States  who  are  dealing  with  vocational  and  technical 
education.  They  use  survey  procedures  and  CODAP-type  programs 
to  ensure  that  vocational  courses  are  on  target  with  civilian 
industry.  The  University  of  Texas  actually  teaches  a  course 
in  CODAP  and  that  will  help  the  academic  world.  Montreal 
Hydro,  the  second  largest  energy-producing  business  in  the  world, 
has  a  strong  CODAP  program.  Control  Development  Corporation, 
a  very  large  computer  manufacturer  in  the  United  States,  is 
using  CODAP  for  job  evaluation  as  a  means  to  job  pricing. 


CONCLUSION 


29.  What  I've  discussed  is  the  method  of  collecting  data 

which  describes  what  individuals  do  on  the  job.  We  know  that 
the  information  is  trustworthy.  We  have  powerful  tools  for 
analysing  this  information  and  reporting  job  descriptions. 

We  can  collect  data  about  the  nature  of  tasks,  such  as  how 
difficult  they  are  and  the  consequences  of  not  doing  them  pro¬ 
perly.  We  can  put  all  of  this  together  to  answer  a  large 
number  of  management  problems. 
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CODAP  IN  THE  OUTSIDE  WORLD 


MILITARY 

INDUSTRIAL 

US  Marine  Corps 

AT&T 

US  Army 

Ontario  Hydro 

US  Navy 

CDC 

US  Coast  Guard 

GOVERNMENT 

Australian  Forces 

NSA 

British  Navy 

DIA 

Canadian  Forces 

0PM  (CSC) 

STATE  AND  LOCAL  AGENCIES 

FAI 

VTECS 

Dept  of  Agriculture 

Prince  George's  County,  MD 

NIH 

Commonwealth  of  Virginia 

Texas  Dept  of  Public  Safety 

Texas  Education  Agency 

Texas  Day  Care  Assn 

UNIVERSITIES 

University  of  Texas 

Texas  A&M 

North  Carolina  State 

John  Hopkins  University 
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HISTORY  OF  JOB  SURVEY 
AND  CODAP  ANALYSIS 
SYSTEM 


CODAP  PROGRAM  SUMMARY 


Listed  below  are  descriptions  of  the  major  computer 
programs  associated  with  the  Air  Force  CODAP  package.  A 
six  character  computer  code  and  title  of  each  program  is  given, 
along  with  a  brief  summary  of  its  capabilities. 

AUTOJT  Automated  Job  Typing 

This  program  evaluates  between-group  differences  for 
pairs  of  Job  Descriptions  to  aid  in  determining 
distinct  job  types  within  the  hierarchical  clustering 
process.  Six  comparisons  are  computed  and  reported 
for  each  pair  of  job  descriptions.  These  evaluations 
include  difference  in  percent  time  on  each  task,  per¬ 
cent  time  spent  on  each  duty,  percent  members  per¬ 
forming  each  task,  number  of  tasks  needed  to  account 
for  a  specified  percent  of  total  group  time,  and 
average  number  of  tasks  performed  by  each  group. 

AVALUE  Average  Values  Per  Task 

AVALUE  computes  an  average  value  for  each  task  in  the 
Job  Inventory.  The  average  value  is  based  on  a 
selected  variable  for  all  cases  who  perform  that  task. 
For  example,  this  program  might  be  used  to  compute 
the  average  number  of  months  in  service  for  those 
members  performing  each  task. 

AVGPCT  Average  by  Percent  Performing 

AVGPCT  computes  an  average  value  for  each  task  in  the 
Job  Inventory.  The  average  is  based  on  the  percent 
of  members  at  each  level  performing.  In  other  words, 
the  average  will  be  adjusted  to  account  for  unequal 
membership  within  each  interval. 

CODAPI  Interface  to  Input  Data  to  CODAP 

The  purpose  of  CODAPI  is  to  provide  an  interface  with 
other  analysis  packages.  This  program  will  take  a 
COBOL-format  file  and  create  CODAP  compatible  data 
cards . 

CODAPX  Interface  to  Extract  Data  from  CODAP 

The  purpose  of  CODAPX  is  to  provide  an  interface  with 
other  analysis  packages.  This  program  will  take  a 
CODAP  History  or  KPATH  file  and  create  a  COBOL-format 
file. 

COMGEN  Composite  Factor  Generator 

COMGEN  allows  the  user  to  generate  a  special  purpose 
FORTRAN  program  to  perform  operations  on  vectors  from 
the  Job  Description  file  and  produce  new  composite 
task  factors. 
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CORREG 

CURVES 

DIAGRM 

DICTXX 

DIST2X 

DUVAKS 

EXTRCT 

FACCOR 


Correlation,  and  Regression  Package 
This  program  extracts  up  to  100  variables  from  a 
CODAP  History  or  KPATH  file  and  computes  correlation 
matrices  and  regression  problems.  The  correlation 
part  computes  and  prints  the  correlation  matrix, 
number  of  valid  cases  in  the  sample,  and  means  and 
standard  deviations  of  the  variables.  A  series  of 
regression  problems  may  be  computed  using  an 
iterative  technique.  The  standard  and  raw  score 
weights  for  each  variable  are  reported,  as  well  as 
the  regression  constants. 

Curve  Fitting  and  Plotting 

This  program  finds  the  curve  of  best  fit  when  predicting 
one  variable  (Y)  from  another  variable  (X)  using 
polynomials.  Provisions  exist  for  plotting  the  curve 
of  best  fit  with  scattergram  of  actual  observations 
superimposed.  At  the  end  of  each  report  a  summary  is 
printed  which  includes  means  and  standard  deviations, 
the  correlation  matrix,  and  a  regression  problem  table 
including  RSQ,  regression  weights  and  constant. 

Diagram  of  Clustering  Process 

This  program  generates  a  treelike  diagram  that  visually 
displays  the  order  in  which  groups  merged  during  the 
hierarchical  grouping  process.  Each  node  of  the  tree, 
representing  one  stage,  displays  the  number  of  members 
at  this  stage,  the  KPATH  range  defining  the  membership, 
and  the  best  and  average  values. 

Print  Variable  Dictionary 

DICTXX  will  provide  the  user  with  a  list  of  variables, 
titles,  and  their  respective  formats  as  defined  on  the 
history  or  KPATH  file. 

2-Way  Distribution 

DIUT2X  reports  a  cross-tabulation  of  values  for  two 
variables,  either  computed  or  background,  for  specified 
cases.  In  addition  to  raw  frequency,  percent  of  total 
row,  total  column,  and/or  total  sample,  mean  and 
standard  deviations  may  be  displayed.  An  additional 
row  or  column  labled  "other"  may  be  added  to  account 
for  values  outside  the  specified  limits. 

Duty  Variable  Computations 

For  each  individual  case,  DUVARS  computes  the  total 
percent  time  spent  in  each  duty,  the  number  of  tasks 
performed  in  each  duty.  They  may  be  stored  as  new 
background  variables  on  the  history  or  KPATH  file. 

Reprinting  of  Reports 

Extract  will  reprint  any  report  or  group  of  reports 
saved  on  the  CODAP  Report  file  during  an  analysis. 

Task  Factor  Correlation 

This  program  will  extract  up  to  100  factors  on  the  CODAP 
FAC3ET  file  and  compute  correlation  matrices  and 
regression  problems.  The  computations  and  reports 
are  like  those  of  CORREG. 
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FACEXT 


FACGEN 


FACPRE 


FACPRT 


FACSTD 


GRMBRS 


GROUP 


Factor  Extract 

FACEXT  will  extract  up  to  100  vectors  on  the  CODAP 
FACSET  file  and  write  them  to  a  COBOL  file .  The 
purpose  of  this  program  is  to  establish  an  interface 
to  other  statistical  packages  available  outside 
CODAP. 

Factor  Generator 

The  purpose  of  FACGEN  is  to  modify  and/or  load  task 
factors  for  future  processing  within  the  CODAP  system. 
Modifying  consists  of  raising  values  to  a  specified 
power,  standardizing  to  a  mean  of  5.0  and  standard 
deviation  of  1.0,  or  the  substitution  of  rescaled  or 
rank  ordered  values. 

Predicted  Factors 

FACPRE  will  apply  the  regression  equations  developed 
by  FACCOR  or  predicted  factors,  the  regression 
equation  including  titles  for  the  input  vectors,  the 
number  of  observations,  product-moment  correlation 
and  product -moment  correlations  squared,  and  the 
standard  error  of  the  estimate.  The  mean,  moment 
about  the  mean,  standard  deviation,  coefficient  of 
variation,  and  minimum  and  maximum  values  are  reported 
in  columnar  format  for  easy  comparison  of  the  criterion 
versus  the  predicted  factor. 

Task  Factor  Print  Program 

This  program  allows  the  user  to  print  any  of  the  factors 
on  the  CODAP  FACSET  file.  Its  capabilities  include 
calculating  and  reporting  differences,  cumulative 
percentages,  categories  of  tasks,  means,  standard  dev¬ 
iations,  and  summations.  Report  formats  may  be 
tailored  in  a  variety  of  ways  to  meet  the  needs  of 
different  users. 

Input  Standard  for  Factor  Raters 

FACSTD  creates  the  CODAP  Rater  History  file.  Its 
specifications  are  similar  to  that  of  INPSTD  except  that 
raw  task  responses  are  stored  instead  of  relative  percent 
time  spent. 

Group  Membership 

This  program  produces  a  detailed  report  which  describes 
the  two  groups  combining  at  each  stage  of  the  hier¬ 
archical  grouping  process.  The  information  reported 
includes:  stage  numbers,  number  of  members  in  the 
combined  group,  KPATH  range  of  the  member  cases,  number 
of  members  in  each  merging  group,  average  overlap 
between  merging  groups,  and  average  overlap  within 
the  combined  group. 

Hierarchical  Clustering 

GROUP  is  the  program  that  actually  performs  the  hier¬ 
archical  clustering  in  the  CODAP  system.  At  every 
stage,  the  two  most  similar  groups  are  identified  and 
combined.  Once  combined,  the  similarity  of  this 
composite  group  with  all  groups  is  reassessed.  This 
collapsing  process  is  continued  until  only  a  single  group 
remains.  The  output  from  GROUP  is  used  by  the  KPATH 
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GRPDIF 

INPSTD 

JOBGRP 

JOBIND 

JOBPRT 

JOBSPC 

J0B3XX 

KPATH 

MODSXX 


program  to  incorporate  the  clustering  information 
back  into  the  mainstream  of  the  CODAP  system. 

Group  Differences 

This  program  will  report  the  task-level  differences 
between  the  two  job  descriptions.  Percent  time 
spent  or  percent  members  performing  each  task  is 
used  as  the  basis  of  computation.  Correlations 
between  the  percent  time  spent  vectors  and  between 
the  percent  members  performing  vectors  may  also  be 
obtained.  That  data  may  be  either  percent  members 
performing  or  average  percent  time  spent  by  all  group 
members. 

Input  Standard  for  Job  Incumbents 

INPSTD  creates  the  standard  CODAP  History  file.  This 
program  stores  percent  time  spent  computed  from  raw 
relative  time  responses  Duty/Task  title  cards  and 
History  variable  definitions  are  combined  with  the 
case  data  and  reorganized  in  History  file  format. 

INPSTD  will  accept  20,000  cases,  1700  task  rat ir.gr, 

6000  characters  of  History  data  per  case,  arid  «>  jjfy 
categories. 

Compute  Stage  Job  Descriptions 

Given  a  stage  number  from  the  hierarchies.  Ci-u.steri.rig 
process,  this  program  identifies  all  members  in  tue 
group  formed  at  that  stage  and  computes  a  composite  jot) 
description  for  those  cases. 

Print  Individual  Job  Description 

This  program  prints  a  job  description  with  specified 
background  information  for  each  indiviuual  in  a  sele¬ 
cted  group. 

Job  Description  Print  Program 

This  program  prints  job  descriptions  computed  by  JOBSPC 
or  JOBGRP.  They  may  be  ordered  by  task,  task  within 
duty,  or  by  modules. 

Compute  Special  Job  Descriptions 

Given  the  membership  criteria  in  terms  of  computed  or 
background  variables,  this  program  identifies  all  cases 
meeting  these  requirements  and  computes  a  composite  job 
description  for  that  group. 

Audit  Job  Description 

JOBSXX  is  designed  to  compute  and  print  job  descriptions 
from  all  cases  on  a  History  or  KPATH  file  in  three 
different  sort  sequences. 

Create  KPATH  File 

This  program  will  resequence  cases  on  the  History  file 
as  defined  in  the  clustering  process. 

Audit  Module  Definitions 

MODSXX  will  print  a  task  listing  ordered  by  module 
categories.  Tasks  not  included  in  any  module  def¬ 
inition  will  be  placed  in  a  module  called  "Tasks  Not 
Referenced"  and  printed  at  the  end  of  the  report. 
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MTXPRT 

OVRLAP 

PLOTIT 

PLTVAL 

PREFAC 

PROGEN 

PRTVAR 

RANSEL 

REXALL 


Print  Overlap  Matrix 

This  program  computes  the  overlap  between  all  pairs 
of  input  composite  job  descriptions  and  reports  these 
values  in  matrix  form.  Overlap  may  be  computed  in 
terms  of  average  percent  time  spent  on  tasks  or  in 
terms  of  the  number  of  tasks  performed  in  common. 

Overlap  of  Response  Patterns 

OVRLAP  calculates  the  similarity  between  all  pairs  of 
cases  on  the  History  file.  The  data  are  arranged  into 
a  matrix  format  for  processing  by  the  GROUP  program. 
History  files  of  7000  cases  or  less  may  be  input  to 
this . 

Plot  a  Task  Factor  Histogram 

This  program  accepts  an  input  task  factor  and  plots  a 
histogram  showing  the  distribution  of  values. 

Plot  Mean  and  Standard  Deviation  Values 

PLTVAL  is  designed  to  produce  a  plotting  of  the  mean 

and  standard  deviations  for  a  given  factor. 

Predicted  Factor  Report 

PREFAG  will  apply  the  regression  equations  developed  by 
TSKCOR  or  CORREG  and  produce  a  task  factor  representing 
this  predicted  factor.  This  program  is  being  replaced 
by  FACPRE.  Input  is  a  job  description  file  rather 
than  a  FACSET  file. 

Program  Generator 

PROGEN  generates  a  FORTRAN  program  from  high-level 
commands  and  standard  FORTRAN  statements  to  perform 
any  operations  on  the  incumbent  data  found  on  the 
History  or  KPATH  file.  Its  primary  use  is  to  add  new 
computed  and/or  history  variables. 

Print  Variable  Values 

PRTVAR  will  print  the  values  of  selected  variables  for 
all  cases  on  the  History  or  KPATH  file .  The  output 
is  often  used  in  the  job-typing  process. 

Random  Case  Selection 

RANSEL  will  produce  a  membership  identification  vector 
useable  as  input  to  the  program  SUBSET.  Given  the 
membership  vector  from  a  current  job  description,  this 
program  will  randomly  select  cases  based  on  a  percentage 
of  the  total  or  a  specific  number. 

Inter-rater  Reliability 

This  program  computes  and  reports  the  average  inter¬ 
rater  reliability  coefficient  of  a  single  rater  and 
the  stepped-up  reliability  coefficient  for  the  total 
group  of  raters.  The  program  is  used  in  conjunction 
with  sets  of  task  ratings  made  by  a  large  number  of 
supervisory  personnel.  REXALL  computes  the  correlation 
of  each  rater's  responses  to  the  grand  mean  vector  on 
those  items  he  or  she  rated.  Other  statistics  are 
available  to  identify  deviating  raters.  The  program 
permits  the  user  to  ignore  these  cases  on  subsequent 
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REXSPC 

SETCHK 

SUBSET 

TASKXX 

TSKCOR 

TSKNDX 

VARGEN 

VARSUM 

VARPCT 


passes.  Also  reported  are  the  mean  task  ratings 
and  the  standard  deviations  of  the  ratings  for  each 
task. 

Special  Task  Factor  Computation 

REXSPC  will  compute  composite  task  factors,  from  rater 
data,  based  on  selected  background  items. 

Check  Sets  of  Raw  Data  Cards 

SETCHK  edits  the  raw  data  which  will  be  input  to  the 
program  INPSTD.  Only  complete  cases  are  kept  for 
further  processing. 

Create  a  Subset  History/KPATH  File 

SUBSET  will  create  a  new  History  file  containing  only 
those  members  as  defined  by  a  composite  job  description 

Duty  and  Task  Title  Print 

TASKXX  is  designed  to  print  a  list  of  the  duty  and  task 
titles  as  entered  in  the  INPSTD  program. 

Task  Factor  Correlations 

TSKCOR  is  being  replaced  by  FACCOR.  Input  is  a  Job 
Description  file  instead  of  a  FACSET  file. 

Task  Index 

TSKNDX  computes  and  prints  the  following  information  for 
tasks  performed  by  a  selected  group  of  members:  task 
titles,  mean  rating  value  ("Task  Index"),  percent  mem¬ 
bers,  and  cumulative  sum  of  average  percent  time  spent 
by  all  members. 

Variable  Generator 

VARGEN  will  compute  new  variables  for  every  case  on  the 
History  or  KPATH  file.  These  variables  are  based  on 
the  individual's  task  response  data.  Some  of  the  value 
computed  are:  average  task  difficulty  per  unit  time 
spent  (ATDPUTS),  overlap  or  an  individual's  time  spent 
with  a  given  job  description,  and  time  spent  over  a 
specified  set  of  tasks.  When  comparing  a  rater  with  a 
given  task  factor  policy,  VARGEN  can  compute  the  sum  of 
absolute  difference,  sum  of  squared  differences,  and 
Pearson  product-movement  correlations. 

Variable  Summary 

VARSUM  computes  and  prints  frequency  distributions  for 
specified  intervals,  reports  total  frequency  counts,  and 
calculates  means  and  standard  deviations  on  selected 
background  and  computed  variables  for  any  group  of 
individuals  whose  job  descriptions  has  been  generated  by 
JOBSPC  or  JOBGRP. 

Variable  Percent  Summary 

VARPCT  produces  the  same  output  as  VARSUM  except  that 
instead  of  reporting  frequency  count*. ,  percentages  are 
used. 
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Variable  Value  Audit  Distributions 

VARSXX  is  designed  to  produce  a  listing  of  the  actual 
responses,  either  history  or  task,  given  by  all  incum¬ 
bents  on  a  History  or  KPATH  file.  This  output  is  very 
useful  for  establishing  specifications  for  the  VARSUM 
and  JOBSAC  programs.  It  is  also  used  as  an  auditing 
tool  of  the  History  file  after  INPSTD. 
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OCCUPATIONAL  ANALYSIS 

SATISFYING  THE  INFORMATION  NEEDS  OF  MANAGERS 
By  Dr.  R.E.  Christal 

INTRODUCTION 


1.  I'd  like  to  start  with  the  statement  that  occupational 

information  does  not  solve  management  problems.  To  the 
contrary,  such  information  often  serves  to  bring  into  sharp 
focus  problems  that  have  gone  unrecognized.  Yet,  detailed  and 
trustworthy  information  about  jobs  and  people  can  be  the  key¬ 
stone  for  good  personnel  management,  and  I  believe  that  a 
manager  armed  with  occupational  survey  data  and  the  CODAP  analysis 
system  is  in  a  position  to  function  effectively. 


APPLICATIONS 


2.  Below  are  listed  some  of  the  important  problem,  areas 

where  managers  can  make  good  use  of  CODAP.  While  the  list  is 
not  exhaustive,  I'm  sure  you  will  recognize  that  it  does  cover 
the  major  components  of  personnel  management.  Throughout  the 
week,  I  will  discuss  most  of  these  application  areas.  During 
this  session,  I  will  give  examples  of  studies  we  have  accomplished 
associated  with  personnel  utilization,  classification  structures, 
and  job  requirements. 


MAJOR  APPLICATION  .AREAS 


Personnel  Utilization 
Job  Requirements 
Skills  Management 
Manpower  Analyses 


Classification  Structures 
Training  Requirements 
Manning 
Job  Evaluation 


Personnel  Utilization 


3.  Personnel  utilization  has  the  most  direct  application 

with  CODAP.  We've  had  questions  such  as  the  following:  Are 
Air  Force  supervisors  discriminating  against  minority  groups  in 
work  assignments?  How  are  women  being  utilized  in  aircraft 
maintenance  jobs?  When  a  person  is  promoted,  how  does  his  job 
change?  Does  he  take  on  new  types  of  activities  appropriate  for 
his  higher  grade  level?  Do  officers  perform  some  tasks  that  would 
be  better  performed  by  enlisted  personnel?  How  do  civilian 
and  military  employees  share  the  work  loads  when  they  are  working 
side  by  side?  What  impact  do  civilians  have  on  the  nature  of 
military  careers?  What  do  people  do  in  the  first  three  months 
on  a  job?  What  are  the  first  activities  that  supervisors  tend 
to  assign  to  new  workers?  What  is  it  about  their  jobs  that  make 
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people  happy  or  unhappy?  Now,  I  don't  know  what  your  specific 
questions  are,  but  if  you  need  information  about  a  subset  of 
individuals,  then  you  have  a  straightforward  application  of 
CODAP  that  can  be  done  quickly  and  efficiently.  I  will  briefly 
describe  four  applications  of  CODAP  to  problems  concerning 
personnel  utilization.  They  relate  to:  discrimination  against 
blacks  in  terms  of  work  assignments;  utilization  of  women  in 
non- traditional  jobs;  utilization  of  low  mental  level  personnel; 
and  the  sharing  of  work  by  nurses  and  medical  service  personnel. 

4.  I'm  sure  that  all  of  you  are  aware  of  the  social  move¬ 
ment  in  the  United  States  which  is  designed  to  assure  equal  and 
fair  treatment  to  all  of  our  citizens,  regardless  of  race, 
religion,  sex,  or  national  origin.  The  military  services  have 
always  led  in  such  movements.  For  example,  the  races  were  housed 
together  by  the  services  long  before  integration  became  a  big 
push  in  the  civilian  sector.  However,  the  question  was  asked 

as  to  whether  discrimination  was  occurring  at  the  lower  levels. 

Do  supervisors  treat  whites  and  blacks  differently?  Do  they 
assign  to  blacks  the  simple  mundane  tasks,  while  giving  the 
more  responsible  and  satisfying  tasks  to  whites.  How  do  you 
address  a  question  like  that?  You  can't  simply  ask  super¬ 
visors  because,  if  you  suspect  a  bias,  you  wouldn't  trust  their 
answers.  Our  approach  was  to  apply  CODAP  to  occupational  survey 
data  to  determine  the  nature  of  work  being  assigned  to  each 
individual,  and  then  to  analyze  this  information  for  bias  in 
work  assignments.  Our  sample  consisted  of  11,000  cases  in  12 
occupations,  23  percent  of  which  were  blacks.  Holding  constant 
such  factors  as  grade  and  time-in-service,  we  used  the  multiple 
linear  regression  model  to  compare  blacks  and  non-blacks  in  terms 
of  the  number  of  tasks  performed,  job  difficulty,  reported 
utilization  of  talent,  and  job  interest.  We  were  able  to  provide 
a  precise  answer  to  the  basic  question  because  of  the  availability 
of  occupational  data  and  the  CODAP  system.  Essentially,  no 
discrimination  was  uncovered.  In  10  of  the  12  occupations 
analyzed,  there  was  no  difference  in  the  nature  of  work  assigned 
blacks  and  non-blacks.  In  two  occupations,  blacks  were  assigned 
slightly  less  difficult  work  than  non-blacks.  However,  in  both 
instances,  the  blacks  reported  higher  interest  in  their  jobs  and 
better  utilization  of  their  talents  than  non-blacks. 

5.  During  the  1960's,  there  was  a  great  experiment  in  the 
United  States  in  which  thousands  of  individuals  were  entered 
into  the  military  services  who  scored  lower  on  mental  tests 
than  existing  cutting  scores.  The  goal  was  to  see  if  the 
services  could  make  good  use  of  them.  There  were  difficulties 
in  getting  this  sample  through  entry-level  courses.  Many 
individuals  had  to  be  given  remedial  training.  However,  even¬ 
tually,  most  of  them  completed  courses  and  were  assigned  to 
jobs.  The  question  was  whether  they  carried  their  load  in  the 
operational  setting.  Supervisory  ratings  of  job  performance 
were  obtained  which  indicated  that  most  of  them  were  performing 
their  jobs  satisfactorily.  But  there  was  a  problem  with  this. 
Suppose  you  had  an  individual  working  for  you  who  didn't  seem 
very  bright.  You  assign  him  the  relatively  mundane  tasks,  and 
he  attacks  them  with  enthusiasm  and  does  an  excellent  job.  How 
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would  you  rate  his  performance?  You  would  probably  indicate 
he  performs  his  job  in  an  excellent  manner.  If  this  example 
were  replicated  throughout  the  services,  we  might  make  a 
mistake  in  evaluating  the  utility  of  lower  mental  level  personnel. 
Management  needed  to  know  not  only  how  well  the  lower  mental 
level  individuals  were  performing  their  assignments,  but  also 
whether  they  were  performing  the  same  difficulty  of  work  as  their 
higher  mental  level  co-workers.  We  were  able  to  analyze 
occupational  data  using  the  CODAP  system  to  answer  this  question. 
We  found  differences  to  be  small,  but  they  were  significant 
in  a  number  of  occupational  areas.  An  interesting  observation 
was  that  low  aptitude  people  could  fill  out  job  inventories. 

The  rejection  rate  on  booklets  that  were  filled  out  by  the  low 
mental  level  group  was  two  percent.  Our  normal  rejection  rate 
for  qualified  people  had  been  about  one  percent.  I  think  one 
of  the  reasons  for  our  success  was  that  we  wrote  task  statements 
in  the  words  of  the  worker,  and  even  a  person  with  a  limited 
vocabulary  does  pick  up  the  vocabulary  of  his  own  occupation. 
That's  one  reason  I  don't  like  to  use  standard  verbs  for  task 
statements.  While  I  am  on  this  topic,  I  should  mention  an  Army 
study  of  low  aptitude  personnel  which  I  think  was  of  consequence. 
They  found  that  the  primary  difference  between  lower  aptitude 
and  higher  aptitude  people  was  not  the  ultimate  level  of 
performance  obtained  from  those  individuals,  but  how  long  it 
took  them  to  reach  a  satisfactory  level  of  skill.  The  lower 
aptitude  people  eventually  got  there,  but  it  took  more  time. 

It  appears  that  a  primary  factor  measured  by  aptitude  tests 
is  the  time  required  for  skill  acquisition.  That's  an 
important  concept. 

6.  One  of  the  ways  the  Air  Force  hopes  to  solve  its  manning 

problem  during  the  1980's  is  by  greater  utilization  of  women. 

I  understand  that  the  manpower  plans  call  for  80,000  women  in 
the  Air  Force  by  1982.  We  are  very  anxious  to  find  out  how 
well  women  will  perform  jobs  of  the  nontraditional  type.  In 
a  recent  probe  study,  we  administered  a  job  inventory  in  the 
Aircraft  Mechanics  career  field.  In  the  sample,  we  had  5825 
males  and  206  females.  The  first  problem  we  encountered  in 
this  study  was  that  one  half  of  one  percent  of  the  mechanics 
misidentified  their  sex.  That's  not  a  high  error  rate,  but  it 
is  serious  because  about  50  men  out  of  approximately  6000  called 
themselves  female.  This  error  had  to  be  corrected.  The  second 
problem  was  that  the  average  time  in  service  by  males  and  females 
differed  by  about  three  months.  In  order  to  make  a  fair 
comparison,  we  took  every  female  and  drew  out  at  random  about 
five  males  with  the  same  lengths  of  service.  This  provided 
two  samples  which  were  comparable  in  terms  of  service  time. 

Next,  we  divided  our  job-types  into  two  classes  -  maintenance 
jobs  and  support  jobs.  The  support  jobs  were  concerned  with 
scheduling,  safety  job  control  and  things  of  that  nature. 

Table  1  shows  the  sizes  of  the  survey  and  analysis  samples. 
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8-43  Months  Total  Active  Federal  Military  Service  (TAFMS) 

Survey  Sample 

Males 

Females 

TAFMS  Number 

1959 

206 

Mean  (months) 

25.8 

22.9 

Std  Deviation 

10.1 

7.3 

8-4?  Months 

Analysis  Sample 

Males 

Females 

TAFMS  Number 

1015 

202 

Mean  (months) 

22.9 

22.9 

Std  Deviation 

7.2 

7.3 

Table  1 .  Selection  of  Sample  for  Utilization  of  Women  Study, 
Aircraft  Mechanics  431  X  1  C,  E,  and  F. 

7.  Table  2  shows  the  distribution  of  cases  across  the 

support  and  maintenance  jobs.  There  was  a  higher  percentage 
of  women  in  support  jobs,  and  a  higher  percentage  of  men  in 
the  maintenance  jobs. 


SUPPORT  JOBS 

MAINTENANCE  JOBS 

Tech  Orders 

Crew  Chiefs* 

Training 

Documentation 

Inspection 

Scheduling 

Special  Maintenance 

Safety 

Job  Control 

Deficiency  Analysis 

Bench  Stock 

CONTAINS 

CONTAINS 

5.65*5  of  males 

26.2#  of  females 

94.4#  of  males 

73.8#  of  females 

-  24  - 


* 59 %  of  all  males  and  44/°  of  all  females  work  in 
"Grew  Chief"  jobs. 


Table  2.  Distribution  of  Males  and  Females  in  Support  and 
Maintenance  Jobs. 


The  next  step  was  to  divide  months  of  service  into  first  year, 
second  year,  third  year  and  beyond.  As  can  be  seen  in  Table  3, 
the  number  of  males  in  the  support  jobs  builds  up  over  time, 
but  not  as  fast  as  for  females.  A  greater  percentage  of  females 
move  from  hard-core  maintenance  to  support  jobs  over  time.  At 
this  rate,  by  the  end  of  four  years,  about  45  percent  of  the 
females  will  be  in  support  jobs.  We  don't  know  why  this  is 
happening;  whether  its  because  they  are  more  talented  for  support 
jobs  (some  of  which  are  quite  difficult)  or  because  they  are 
unable  to  perform  the  maintenance  tasks.  As  I  said  before,  some¬ 
times  all  you  can  do  is  identify  the  problem. 


TAFMS 

%  OF  MALES 

%  OF  FEMALES 

0-12 

3.74 

8.70 

13-24 

3.69 

17.92 

25-43 

8.65 

43.84 

TOTAL 

5.62 

26.24 

Table  3.  Percentage  of  Males  and  Females  in  "Support"  Job 
Types  by  TAFMS. 


8.  Next,  we  classified  the  tasks  into  certain  categories 

and  computed  the  information  in  Table  4.  The  women  in  the 
maintenance  job  types  are  performing  work  very  similar  to  men. 
Notice  that  they  are  performing  both  "heavy"  tasks  and  "dirty" 
tasks.  However,  there  are  considerable  differences  in  the  work 
performed  by  men  and  women  in  the  support  jobs.  The  data  shown 
in  Table  5  are  even  more  revealing.  Approximately  74  percent  of 
the  women  and  94  percent  of  the  men  were  assigned  to  the  "hard 
core"  maintenance  job  types.  Within  these  job  types,  there  is 
very  little  difference  in  the  duties  performed  by  the  two  sexes. 
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Class  of  Tasks 

Support 

Maintenance 

Male 

Female 

Male 

Female 

Clerical 

26.4 

46.7 

5.6 

6.8 

Heavy  Maintenance 

2.5 

.3 

8.6 

8.3 

Light  Maintenance 

5.6 

1.2 

14.6 

13.9 

"Dirty"  Maintenance  Tasks 

.8 

.6 

8.6 

8.3 

Inspect,  Check,  Troubleshoot 

8.6 

3.2 

37.2 

37.2 

Other  (Support,  Non-Clerical) 

56.0 

48.1 

25.3 

25.4 

AVG  NO  OF  TASKS  PERFORMED 

28.1 

18.8 

157.9 

141.1 

AVG  TASK  DIFF.  PER  UNIT  TIME 

4.4 

5.0 

4.4 

4.3 

Table  4.  Percentage  Time  Spent  on  Various  Classes  of  Tasks 
by  Men  and  Women  in  "Support"  v  "Maintenance"  Job 
Types. 


Duty 

— 

1o  Time 
Male 

io  Time 
Female 

io  Time 
Difference 

G  Performing  General  Aircraft 
Maintenance 

22.61 

22.53 

0.08 

H  Performing  Ground  Handling  of 
Aircraft 

20.03 

21  .63 

-1  .60 

I  Maintaining  Landing  Gear 
Systems 

11.24 

9.88 

1  .36 

M  Maintaining  Electrical 

Systems 

6.32 

7.27 

-0.95 

K  Maintaining  Flight  Control 
Systems 

5.19 

4.56 

0.63 

Q  Performing  General  Engine 
Maintenance 

5.05 

4.43 

0.62 

L  Maintaining  Pneudraulic 
Systems 

5.00 

5.37 

-0.37 

E  Maintaining  Forms  and  Records 

4.79 

5.55 

-0.76 

0  Maintaining  Non-Powered  AGE 
Equipment 

4.45 

3.80 

0.65 

N  Maintaining  Fuel  Systems 

4.17 

4.04 

0.13 

J  Maintaining  Utility  Systems 

3.23 

3.H 

0.09 

F  Performing  Supply  Functions 

2.46 

2.51 

-0.05 

C  Inspecting  and  Evaluating 

1 .54 

1 .41 

0.13 

Subtotals 

96.08 

96.12 

Table  5.  Percentage  Time  on  Various  Duties  for  Male  and  Female 
Personnel  Working  in  Maintenance  Job  Types. 
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9.  We  took  this  analysis  a  little  further.  There  were 

111  tasks  performed  hy  men  which  no  women  were  doing.  Actually, 
the  men  spent  only  a  fraction  of  their  time  on  those  tasks.  We 
analysed  the  tasks  in  terms  of  difficulty,  lifting  requirements, 
and  so  on,  hut  found  nothing  significant.  We  concluded  that 
women  could  perform  these  tasks,  but  we  identified  none  because  of 
sampling  error.  We  simply  failed  in  our  small  sample  to  pick 
up  women  performing  these  tasks.  Finally,  as  a  summary,  we 
correlated  time  spent  across  all  the  tasks  in  the  inventory. 

(See  Table  6)  We  can  say,  with  some  assurance,  that  there  is 
no  difference  in  the  work  performed  by  men  and  women  in  main¬ 
tenance  job  types.  Perhaps  the  most  consequential  thing  to 
come  out  of  the  study  concerns  aptitude  scores.  Women  scored 
so  low  on  the  mechanical  tests  that  we  had  to  waiver  the 
mechanical  aptitude  requirement  for  maintenance  workers.  You 
will  see  in  Table  7  the  percentile  scores  for  males  and  females. 
The  mean  score  for  males  is  68  and  the  mean  score  for  females 
is  39.  That’s  a  tremendous  difference.  Women  scored  very 
poorly  on  mechanical  aptitude  tests;  yet  we  discovered  that  in 
the  maintenance  area,  women  were  performing  the  same  tasks 
as  men.  We  looked  at  the  contents  of  the  tests  and  found  that 
they  were  oriented  toward  mechanical  experience.  There  were 
questions  about  the  recognition  of  shop  tools  and  questions 
about  automotive  mechanics  which  men  have  been  exposed  to  all 
their  lives.  But  many  women  had  no  previous  exposure  to  those 
experiences.  We  believe  now  that  we  need  a  different  type  of 
mechanical  test  for  women  which  is  not  experience  bound. 


Variables 

Correlation: 

Male  vs  Female 

io  Time  spent  on  tasks  in  MAINTENANCE  Job  Types 

.97 

io  Performing  tasks  in  MAINTENANCE  Job  Types 

.99 

i  Time  spent  on  tasks  in  SUPPORT  Job  Types 

.69 

i  Performing  tasks  in  SUPPORT  Job  Types 

.58 

Table  6.  Correlations  Between  Male  and  Female  Work  in  431x1 
C,  E  &  F. 


Aptitude 

Composites 

Mean 

Male 

Mean 

Female 

Mechanical 

68.9 

39.2 

Electronic 

64.5 

60.7 

General 

60.7 

69.9 

Administrative 

47.6 

63.4 

Table  7.  Aptitude  information  for  Analysis  Sample  431x1 
Utilization  of  Women  Study. 
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10.  A  few  years  ago,  the  Surgeon  General  and  the  Chief 
of  Nurses  were  concerned  about  the  shortage  of  trained  nurses. 
They  wanted  to  know  the  extent  to  which  the  work  of  trained 
nurses  and  medical  services  specialists  overlapped.  We  built 

a  multi-occupational  inventory  encompassing  all  the  tasks  known 
to  be  performed  by  nurses  and  enlisted  medical  services 
personnel.  After  the  survey,  we  were  able  to  perform  a  direct 
analysis  of  the  tasks  which  are  shared  by  the  two  groups.  We 
found  considerable  overlap  in  the  work  performed.  There  were 
some  differences  in  the  way  people  were  distributed  across  job- 
types,  but  it  appeared  that  additional  medical  services 
personnel  could  reduce  the  requirement  for  nurses.  The  work 
done  by  enlisted  people  was  not  mundane.  As  a  matter  of  fact, 
tasks  like  minor  surgery  were  performed  more  often  by  enlisted 
personnel  than  by  nurses. 

11.  In  another  interesting  study,  we  asked  medical  services 
technicians  to  design  their  own  jobs.  They  were  instructed  to 
check  the  tasks  that  would  be  in  the  job  and  to  allocate  work 
time  to  those  tasks.  The  technicians  were  very  realistic;  they 
put  into  their  job  description  things  like  emptying  bed  pans 
because  they  recognized  it  as  a  task  that  had  to  be  done. 


Classification  Structures 

12.  Decisions  on  how  to  fit  jobs  into  occupational  categories 
are  very  consequential.  They  determine  the  cost  and  the 
effectiveness  of  training  programs  because  you  have  to  train 
people  for  the  entire  occupational  category.  Classification 
decisions  also  determine  the  flexibility  of  personnel  utiliz¬ 
ation  and  the  breadth  of  people's  careers.  If  you  have  a 
narrowly  prescribed  category,  you  know  what  a  man  can  do  after 
eight  years  experience.  On  the  other  hand,  in  a  broad  category 
with  many  different  job- types,  you  may  not  know  what  direct 
experience  each  person  has  had.  In  Australia,  the  classification 
of  categories  has  a  bearing  on  pay,  and  that  is  of  great 
consequence . 

13.  Again,  I  must  point  out  that  information  from  a  CODAP 
job-+ype  analysis  does  not  prescribe  a  solution.  It  will 
describe  the  problem  and  provide  a  lot  of  associated  inform¬ 
ation  for  the  people  in  the  classification  business.  The  CODAP 
system  will  indicate  how  many  individuals  are  in  each  job- 
type  under  consideration  and  where  they  are  located.  This 
information  is  important  to  avoid  creating  new  occupations  only 

to  discover  that  half  the  jobs  are  overseas  or  in  remote  locations. 

14.  Jobs  that  are  put  together  into  an  occupation  need  to 
be  somewhat  homogeneous  in  terms  of  job  requirements,  otherwise 
some  jobs  could  be  performed  by  people  who  are  not  very  bright 
while  other  jobs  would  require  very  talented  people.  In  a  case 
like  this,  if  you  set  the  requirements  for  the  most  difficult 
job,  then  you  would  waste  the  talents  of  the  people  doing  the 
simpler  jobs.  If  you  survey  an  occupation  once,  you  get  a 
static  picture;  if  you  survey  at  two  points  in  time,  you  get  a 
dynamic  picture  which  tells  you  how  many  people  move  from  one 
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job-type  to  another  during  a  given  time  period.  We 
discovered  that  in  some  of  our  occupations,  people  were  staying 
in  the  one  job-type  through  a  major  part  of  their  career.  This 
means  that  a  lot  of  the  training  given  to  these  individuals  was 
never  utilized. 

15.  When  the  US  Marine  Corps  adopted  CODAP  technology, 
they  decided  that  their  classification  structure  was  too  broad. 
This  is  part  of  the  debate  about  specialization  and  general¬ 
ization.  Training  is  easier  if  the  structure  is  highly 
specialized,  but  there  is  less  flexibility  in  the  personnel 
utilization.  The  Marine  Corps  used  the  CODAP  job-type  analysis 
system  in  the  redesign  of  their  classification  structures  and 
received  a  Presidential  Award  for  management  improvement  as  a 
result  of  this  exercise. 

16.  I  must  admit  that  occupational  data  is  better  for 
splitting  occupations  than  for  merging  them.  You  can  use  a 
multi-occupational  inventory  to  answer  precisely  the  extent  to 
which  people  in  two  occupations  are  doing  the  same  tasks,  and 
you  can  look  at  job  requirement  data  to  see  if  aptitude  levels 
and  physical  strength  requirements  are  the  same  for  the  two 
occupations.  The  question  one  can't  answer  directly  is  how 
easily  people  in  one  occupation  can  learn  to  perform  the  tasks 
in  the  other  occupation  being  considered  for  meger. 

17.  You  might  be  interested  in  a  recent  change  in  the  USAF 
classification  structure  which  involved  the  creation  of  new 
Senior  Manager  positions.  Evidently,  there  were  complaints  that 
people  at  the  top  grade  level  (E9)  were  not  being  fully  utilized. 
The  Chief  of  Staff  decided  that  we  should  look  at  the  possi¬ 
bility  of  combining  occupations  at  the  top  levels.  That  meant 
that  Senior  Managers  would  handle  multi-occupations.  We 
produced  for  the  classification  people  the  top  grade  job 
descriptions  for  all  the  occupations  for  which  we  had  data.  We 
discovered  that,  in  most  occupations,  E9's  do  work  that  is 
managerial  in  nature.  They  help  draft  the  budget;  they  arrange 
space  for  people;  they  assign  people  to  work  stations;  and  they 
coordinate  activities  with  other  organizations.  In  most  cases, 
they  do  not  perform  the  work  tasks  that  the  journeymen  perform. 
However,  in  some  occupations,  some  of  the  more  complicated 
technical  tasks  were  being  performed  by  E9's.  From  this 
information,  the  classifiers  were  able  to  determine  where  they 
could  collapse  two  occupations  because  all  of  the  duties  were 
managerial.  The  Director  of  Classification  Structures  Branch 
said  that  he  was  very  pleased  with  the  data  because  several 
proposed  recommendations  would  have  caused  serious  problems  if 
they  had  been  implemented.  The  occupational  information  avail¬ 
able  enabled  him  to  make  good  decisions. 

18.  In  one  study,  we  were  able  to  demonstrate,  using  the 
CODAP  system,  that  the  classification  structures  for  the 
Accounting  and  Disbursement  Career  ladders  were  allowing 
technically  incompetent  individuals  to  be  placed  into  super¬ 
visory  positions.  The  problem  was  generated  by  the  fact  that 
the  two  ladders  were  separated  at  the  apprentice  and  journeyman 
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levels,  but  merged  at  the  first-line  supervisory  level.  Thus, 
it  was  possible  for  an  enlisted  man  to  spend  the  first  part  of 
his  career  in  the  disbursement  ladder  and  end  up  being  a  super¬ 
visor  of  individuals  in  an  accounting  shop.  Ke  would  find 
himself  in  the  position  of  having  to  certify  the  accuracy  of 
accounts,  having  no  idea  concerning  how  they  were  developed. 
Using  the  task  inventory,  we  were  able  to  get  agreement  from 
supervisors  concerning  "critical  tasks"  than  an  individual 
should  not  be  asked  to  supervise  unless  he  has  had  direct  ex¬ 
perience  in  performing  the  task.  We  then  conducted  an 
occupational  survey  of  supervisors  to  determine  their  experi¬ 
ences  on  such  critical  tasks.  The  data  clearly  indicated  that 
the  classification  structures  were  generating  serious  problems, 
so  the  Disbursement  and  Accounting  ladders  were  separated 
through  the  first-line  supervisory  level. 


Job  Requirements 

19.  Job  requirements  can  be  classified  as  physical,  mental, 

and  psycho-physical.  I'll  discuss  the  use  of  CODAP  in 
establishing  mental  requirements  in  a  later  paper.  For  the 
moment,  I'll  address  the  physical  requirements.  The  Air  Force 
is  concerned  about  the  enlistment  of  a  large  number  of  women. 
Although  there's  an  overlap  in  the  distribution  of  strength 
between  men  and  women,  women  as  a  group  don't  have  as  much 
upper  torso  strength  as  men.  The  question  was  whether  our 
occupational  data  bank  could  be  used  to  help  determine  the 
stamina  and  strength  requirements  for  the  various  occupations. 
The  goal  was  not  sex  oriented,  since  any  screening  system  would 
be  designed  to  weed  out  both  men  and  women  not  having  suffi¬ 
cient  strength  to  perform  the  required  work.  Our  goal  was  to 
determine  the  strength  and  stamina  requirements  for  all  tasks 
in  the  Air  Force.  The  study  was  straightforward  because  of  the 
availability  of  task  inventories  and  the  CODAP  analysis  tech¬ 
nology.  First,  we  had  supervisors  in  each  occupation  review 
task  lists  and  check  those  tasks  which,  in  their  judgement, 
yields  variance  in  performance  as  a  function  of  strength  or 
stamina.  They  also  wrote  in  any  demanding  tasks  which  were  nit 
in  the  inventory.  In  the  second  step,  tasks  identified  as 
requiring  significant  strength  or  stamina  were  rated  on  scales, 
such  as  the  one  presented  in  Table  8  .  The  third  step  involved 
collection  of  additional  information  about  requirements,  such  as 
whether  they  involved  carrying,  pushing,  lifting,  etc.  The 
fourth  step,  which  is  about  to  be  executed,  will  involve  on¬ 
site  validation  of  the  ratings  through  physical  measurements 
for  a  sample  of  tasks.  If  this  is  successful,  tests  of  strength 
and  stamina  will  be  developed  to  measure  individual  capacities, 
and  individuals  will  be  assigned  to  jobs  only  if  they  can  meet 
the  defined  strength  and  stamina  requirements.  It  should  be 
noted  that  the  availability  of  task  inventories  and  the  CODAP 
system  make  studies  of  job  requirements  feasible.  In  the  near 
future,  we  hope  to  accomplish  a  similar  study  :o  identify  the 
psychomotor  and  perceptual  requirements  for  Aii  Force  Tasks. 
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SCALE  VALUE 

DESCRIPTOR 

LIFTING  BENCHMARK 

0 

No  Sig  Demand 

0-9  lbs  OR  EQUIVALENT  EFFORT 

1 

Extremely  Light 

1 0-1 9  lbs 

2 

Very  light 

20-29  lbs  " 

3 

Light 

30-39  lbs 

4 

Light  to  Moderate 

40-49  lbs  " 

5 

Moderate 

50-59  lbs 

6 

Moderate  to  Heavy 

60-69  lbs  " 

7 

Heavy 

70-79  lbs 

8 

Very  Heavy 

80-89  lbs  " 

9 

Extremely  Heavy 

90  lbs  OR  MORE 

Table  8.  Physical  Demand  Scale. 


CONCLUSION 

20.  I  have  mentioned  only  a  few  ways  that  occupational 

information  can  be  used  to  help  solve  management  problems.  I 
hope  to  describe  many  other  applications  during  the  remaining 
part  of  this  symposium. 


A 
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OCCUPATIONAL  ANALYSIS  AND  ELECTRONIC  THEORY  TRAINING: 

THE  PAYOFF  FOR  MANAGERS 


Captain  Charles  D.  Gorman,  PhD,  MA( Psych) ,BSc,  USAF 
Headquarters  Support  Command  (RAAF) 

i 


INTRODUCTION 


1 .  The  Royal  Australian  Air  Force  (RAAF)  conducts  occupational 

analysis  from  the  Ground  Training  Division  at  Headquarters  Support 
Command  (HQSC).  In  analyzing  occupational  survey  data,  the 
Comprehensive  Occupational  Data  Analysis  Programmes  (CODAP)  are  used 
in  conjunction  with  analysis  procedures  researched  and  developed  by 
the  United  States  Air  Force  (USAF) .  This  method  normally  employs  the 
task  as  its  basic  unit  of  analysis.  However,  in  1975  the  USAF 
Occupational  Measurement  Centre  developed  a  variation  of  this 
procedure  using  knowledges  in  the  form  of  electronic  principles  as 
well  as  tasks  as  basic  units  of  analysis.  The  resulting  survey 
instrument  was  to  become  known  as  the  Electronic  Principles  Inventory, 
or  EPI  for  short.  In  1977  the  RAAF  became  interested  in  evaluating 
electronics  training  and  so  developed  the  RAAF  version  of  the  EPI. 


AIM 


2.  The  aim  of  this  paper  is  to  familiarize  the  reader  with  the 

EPI  and  its  use  in  developing  and  maintaining  a  programme  of  cost- 
effective,  timely  electronics  training.  More  specifically,  the 
philosophical  background  and  historical  development  of  the  EPI,  both  in 
the  USAF  and  the  RAAF,  will  be  discussed;  examples  of  EPI  data  obtained 
from  the  RAAF  will  be  presented;  and  precautions  for  employing  results 
of  survey  data  in  the  training  environment  will  be  discussed. 


PHILOSOPHICAL  BACKGROUND 


3.  In  the  middle  1970s,  in  the  area  of  electronics  training,  the 

USAF  was  spending  some  $500,000  a  day  in  preparing  men  and  women  to 
take  one  of  63>000  positions  involving  the  use  of  electronic  principles. 
Such  an  expenditure  was  necessary  to  support  the  philosophy  of  training 
which  was  in  vogue  at  the  time:  When  in  doubt  as  to  the  utility  of 
training,  give  it  to  the  student  just  to  be  on  the  safe  side.  In  other 
words,  the  attitude  was  that  extra  training  certainly  wouldn't  hurt, 
and  it  served  to  virtually  insure  that  course  graduates  would  receive 
training  on  whatever  they  might  later  encounter. 
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4.  Nevertheless,  there  are  serious  problems  with  this  philosophy 
of  overtraining.  First,  overtraining  may  engender  in  the  student  a 
set  of  expectations  about  what  his  job  will  be  like  when  he  completes 
his  training.  The  logical  assumption  for  the  student  to  make  is  "I 
must  be  going  to  do  this,  or  they  wouldn't  be  training  me  to  do  this." 

If "this"  is  particularly  interesting  or  exciting  to  the  student,  then 
he  begins  to  build  a  set  of  expectations  about  what  his  job  will  be 
like;  and  if,  when  he  gets  into  the  field,  those  expectations  are  not 
fulfilled,  his  reaction  could  be  detrimental  to  the  service.  For 
example,  he  might  feel  that  he  has  been  conned,  and  thus  resent 

the  service  and  everything  associated  with  it.  Or,  he  may  accept  the 
lack  of  correlation  between  his  training  and  his  job  and  carry  on  in 
some  state  of  confusion.  But  even  more  likely  is  the  prospect  of  his 
reacting  with  disappointment  and  dissatisfaction  with  a  job  that  does 
not  fulfill  his  expectations:  expectations  which  he  logically  formed 
but  which  in  his  view  have  remained  illogically  unfulfilled.  What 
compounds  the  problem  of  dissatisfied  airmen  is  the  probability  that, 
because  they  are  dissatisfied,  they  will  leave  the  service  at  their 
earliest  opportunity,  taking  with  them  their  experience  and  some  very 
expensive  training. 

5.  This  brings  us  to  the  second  major  problem  with  overtraining- 
the  problem  of  cost.  Training  in  the  area  of  electronics  is  expensive 
particularly  in  terms  of  the  equipment  necessary  to  provide  that  training. 
And  because  ability  and  knowledge  in  electronics  are  marketable  attri¬ 
butes  in  the  civilian  community,  costs,  in  the  form  of  replacement 
training,  are  increased  as  significant  percentages  of  trained  personnel 
leave  the  service  for  civilian  jobs.  In  these  days  of  rising  costs 

and  scarce  training  dollars,  we  can  no  longer  enjoy  the  "luxury" 
of  overtraining.  It  makes  little  economic  sense  to  train  100  percent 
of  the  airmen  in  a  mustering  on  technical  tasks  that  only  10  or  15 
percent  of  them  will  be  called  upon  to  perform;  and  unfortunately, 
what  is  trained  is  becoming  more  and  more  influenced  by  economics. 

6.  Against  this  backdrop  of  increasing  costs  and  scarce  training 
dollars,  the  USAF  in  1974  was  forced  to  ask  the  question,  "Is  our 
electronic  principles  training  cost  effective?"  In  other  words,  are 
airmen  using  the  electronic  principles  they  have  been  taught,  or  do 
many  of  these  principles  fall  into  the  category  of  "nice  to  know 
information"  which  usually  bears  little  effect  on  their  ability 

to  perform  their  day  to  day  jobs?  It  was  this  question  which  gave 
birth  to  the  FPI. 


DEVELOPMENT  OF  THE  USAF  EPI 


7.  The  EPI  is  a  means  of  determining  what  electronic  principles 

are  being  used  by  various  groups  of  airmen  in  the  field.  It  accomplishes 
this  by  asking  airmen  to  answer  "yes"  or  "no"  to  a  series  of  questions 
concerning  the  electronic  principles  they  are  using  in  the  performance 
of  their  jobs.  It  differs  from  the  ordinary  occupational  survey  in  two 
important  ways.  First,  the  EPI  asks  two  general  questions:  1)  What  do 
you  do?  and  2)  What  electronic  knowledge  do  you  use  to  do  it? 

Traditional  occupational  surveys  only  ask  the  first  of  these  two 
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questions.  Second,  the  EPI  may  be  administered  to  any  specialty 

or  mustering,  where  ordinary  occupational  surveys  are  usually  mustering- 

specific. 

8.  The  EPI  was  developed  by  the  USAF  Occupational  Measurement 
Centre  in  three  distinct  phases.  Beginning  in  late  1974,  a  detailed 
review  of  literature  concerning  electronics  training  was  conducted. 

This  review  gave  the  inventory  developers  a  broad  overview  of 
technical  information  which  they  could  draw  upon  in  the  development 
of  the  first  draft  of  the  EPI.  That  draft  contained  574  items  which 
represented  some  40  percent  of  electronic  principles  then  currently 
being  trained  in  the  USAF. 

9.  After  field-testing  this  draft  inventory  in  five  USAF 
specialities,  full  scale  development  of  the  EPI  began.  Ten  special¬ 
ists  in  electronics  training  spent  three  weeks  reviewing,  revising, 
and  adding  items  to  the  draft  EPI  developed  in  Phase  I.  The 
resulting  document  consisted  of  1257  electronic  principles  items 
which  were  grouped  under  62  different  headings  or  subject  matter 
areas.  These  62  areas  covered  all  electronic  principles  then 
currently  being  trained  by  the  various  USAF  technical  training 
schools . 

10.  After  field-testing  this  final  version  of  the  EPI,  a  full-scale 
administration  to  all  members  of  a  USAF  technical  occupational  area 

was  undertaken.  Results  of  this  survey  and  the  previous  field  tests 
indicated  considerable  variation  across  individuals  and  groups  of 
individuals  with  respect  to  the  number  of  electronic  principles 
required  in  the  performance  of  their  jobs.  Some  specialties'  members 
used  large  numbers  of  principles,  while  others'  used  only  comparatively 
few  principles  in  the  performance  of  their  jobs.  These  data -offered 
considerable  encouragement  to  those  who  were  seeking  a  means  of 
validating  electronics  training  in  the  USAF,  and  the  EPI ' s  potential 
applicability  in  other  services  was  quickly  recognised. 


DEVELOPMENT  OF  A  RAAF  EPI 


11.  In  1977,  the  situation  with  respect  to  electronics  training 

in  the  RAAF  was  recognised  as  being  similar  to  that  in  the  USAF. 

In  a  letter  from  Air  Force  Office  to  Staff  Officer  Ground  Training 
at  HQSC,  increasing  complexity  of  equipment  and  escalating  costs 
of  training  and  manpower  were  cited  as  major  factors  in  bringing 
to  the  forefront  the  need  to  look  critically  at  the  electronics 
training  then  being  offered  to  various  RAAF  musterings,  specifically 
six  technical  trades  trained  at  the  RAAF  School  of  Technical  Training 
(RAAFSTT),  Wagga.  Coincidentally,  Air  Force  Office  received  a  copy 
of  a  recently  completed  USAF  EPI  report.  The  combination  of  these 
factors  culminated  in  a  request  from  Air  Force  Office  that  the 
GT  division  undertake  a  project  similar  to  the  USAF  EPI,  with  the 
ultimate  aim  of  validating  existing  electronics  training  in  the  RAAF. 
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12.  Accordingly,  a  technical  expert  in  the  area  of  electronics 

was  attached  to  the  Occupational  Analysis  and  Training  Evaluation 
Section  of  the  Ground  Training  Division  at  HQSC  for  a  six  week  period. 
During  that  time  an  EPI  patterned  after  the  USAF  EPI  was  drafted  and 
was  reviewed  by  officials  and  technical  experts  at  RAFFSTT.  The 
survey  instrument  was  field  tested  at  RAAF  Base  Laverton,  with 
members  from  all  six  musterings  to  be  surveyed  participating. 

The  resulting  instrument  contained  1039  items  organized  under  23 
headings. 


13.  Once  construction  and  validation  of  the  inventory  were 

completed,  the  survey  instrument  was  administered  to  all  airmen  in  the 
ranks  of  Aircraftsman  (AC),  Leading  Aircraftsman  (LAC)  and  Corporal 
(CPL)  in  the  Engine  Fitter  (ENGFITT),  Instrument  Fitter  (INSTFITT), 
Electrical  Fitter  (ELECFITT),  Motor  Transport  Fitter  (MTFITT), 

Armament  Fitter  (ARMFITT)  and  Air  Frame  Fitter  (AFFITT)  musteringe. 
These  data  were  collected  from  October  -  December  1977.  Results 
of  the  survey  (to  be  discussed  in  general  later  in  this  paper)  were 
so  gratifying  in  terms  of  their  potential  utility  that  another 
Electronic  Principles  Survey,  this  time  of  three  Radio  musterings, 
was  requested  immediately  by  Air  Force  Office.  Since  the  original 
EPI  had  been  developed  to  cover  those  principles  trained  at  RAAFSTT, 
additions  to  the  survey  instrument  were  needed  to  cover  additional 
principles  trained  at  the  RAAF  School  of  Radio  (RADS).  These  were 
obtained  through  the  efforts  of  several  subject  matter  specialists  at 
RADS,  and  the  resulting  modified  EPI,  containing  1509  items  grouped 
under  26  headings,  was  ready  for  administration.  Members  of  the 
Radio  Technician  Air  (RADTECHA) ,  Radio  Technician  Ground  (RADTECHG) 
and  Telecommunications  Technician  (TELSTECH)  musterings  in  the  ranks 
of  AC,  LAC  and  CPL  were  administered  the  survey  during  September  - 
December  1978  with  Survey  results  again  demonstrating  great  potential 
utility. 


SAMPLE  DATA  FROM  THE  RAAF  EPI 


14.  Data  obtained  from  the  two  electronic  principles  surveys 
generally  show  the  percentages  of  any  group  of  interest  that  make 
use  of  or  perform  the  various  principles  or  tasks.  Those  groups 
that  are  generally  of  most  interest  are  either  the  mustering  as  a 
whole  or  the  specific  ranks  within  a  mustering.  In  this  section 

we  will  look  at  some  examples  of  how  these  data  may  be  used  to  address 
specific  problems  within  the  training  environment. 

Commonality  Between  Musterings 

15.  One  question  that  was  implied  in  the  initial  request  for  the 
development  and  administration  of  the  EPI  in  the  RAAF  was,  "Are 
there  sufficiently  large  numbers  of  principles  employed  in  common 
across  musterings  to  justify  common  electronic  principles  training 
for  those  musterings?"  Tables  1  and  2  give  examples 

of  how  EPI  data  can  be  used  to  address  this  question.  In  Table  1 ,  a 
great  degree  of  similarity  of  principles  usage  is  demonstrated  between 
the  RADTECHA  and  RADTECHG  musterings  in  the  general  area  of  pulse 
techniques.  In  Table  2,  however,  little  similarity  is  demonstrated 
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between  RADTECHS  and  TELSTECHS  in  their  usage  of  principles  from 
the  area  of  navigational  aids  and  clippers  and  clampers. 


Principle 

fiBBB 

GENERAL  AREA:  PULSE  TB 

IHNIQUES 

Refer  to  pulse 

60 

60 

Refer  to  pulse  duration 

55 

55 

Refer  to  pulse  width 

62 

60 

Refer  to  pulse  repetition  time 

59 

57 

Refer  to  pulse  period 

48 

45 

Refer  to  pulse  repetition  note 

53 

50 

Refer  to  pulse  interval 

49 

48 

Refer  to  pulse  duty  cycle 

51 

45 

Refer  to  pulse  space  ratio 

38 

47 

Refer  to  pulse  width  modulation 

20 

26 

Refer  to  pulse  width  demodulation 

16 

22 

Table  1 .  Example  of  High  Similarity  Between  Musterings 

Principle 

f>  Using, 
RADTECHG 

%  Using, 
TELSTECH 

GENERAL  AREA:  NAV  , 

\ID5 

Use  or  refer  to  single  current  signalling 

2 

89 

Use  or  refer  to  double  current  signalling 

2 

86 

Use  or  refer  to  waveshapes  of  telegraph 

impulses 

1 

84 

Use  or  refer  to  characteristic  distortion 

0 

63 

Use  or  refer  to  effects  of  live  resistance 

1 

56 

GENERAL  AREA :  CLIPPERS 

AND  CLAMPERS 

Required  to  understand  function  of  a 

positive  clipper 

69 

27 

Required  to  understand  function  of  a 

conducting  valve 

61 

36 

Required  to  understand  function  of  zener 

diodes  in  clippers 

56 

30 

Refer  to  function  of  a  clamper 

58 

18 

Refer  to  the  term  positive  or  negative 

clampers 

57 

17 

Table  2.  Examples  of  Areas  of  Little  Similarity  Between 
Musterings. 


-  36  - 


16.  In  point  of  fact,  the  general  conclusion  drawn  from  the  two 
electronic  principles  surveys  conducted  in  the  RAAF  is  that  there  are 
insufficient  numbers  of  principles  employed  in  common  across  all 
nine  masterings  surveyed  to  justify  a  cost-effective  common  course 

in  electronic  principles.  However,  if  no  one  considers  a  subset  of 
must  rings  (for  example  RADTECHA  and  RADTECHG-,  or  INSTFITT  and 
ELECPITT),  then  there  may  be  grounds  for  common  training  in  some 
principles.  The  USAF  is  currently  investigating  the  possibility  of 
developing  a  set  of  common  electronic  principles  courses,  each  to  be 
attempted  by  members  of  certain  musterings.  With  the  electronic 
principles  data  now  available,  the  RAAF  can  do  the  same. 

Timing  of  Training 

17.  In  much  of  our  military  training,  there  has  been  at  least 
implied  the  idea  that  "we  only  get  our  airmen  for  training  one  time, 
so  let's  give  him  everything  we  can."  There  are  at  least  two  pitfalls 
with  this  approach  to  the  training  of  airmen.  The  first  has  to  do 
with  cost,  and  has  already  been  discussed  earlier  in  this  paper.  The 
second  is  concerned  with  the  basic  premise  of  this  philosophy  -  that 
airmen  are  only  available  for  training  one  time. 

18.  Should  airmen  receive  only  one  course  of  technical  training 
in  electronic  principles?  The  answer  to  this  question  depends  on 
several  factors,  but  the  most  important  concern  should  be  when  the 
individual  is  likely,  with  some  substantial  degree  of  probability, 
to  make  use  of  the  training.  There  may  be  tasks  performed  or 
principles  employed  by  large  percentages  of  young  airmen  shortly  after 
completion  of  training,  and  these  should  obviously  receive  attention 
in  the  basic  technical  training  course.  But  what  about  those  tasks 
and  principles  that  are  not  performed  or  used  by  very  many  lower- 
ranking  people  at  all,  but  rather  are  reserved  for  use  by  more 
senior  personnel?  Training  new  recruits  in  a  basic  course  on  tasks 
and  principles  that  they  will  not  use  until  much  later  in  their  careers 
is  not  only  expensive  (they  may  exit  the  service  before  they  get  a 
chance  to  use  the  training;,  but  it  may  be  just  plain  foolhardy, 
particularly  in  a  highly  technical  area  like  electronics.  In  the 
first  place,  there  is  every  likelihood  that  the  individual  will 
forget  many  technical  aspects  of  training  received  years  before  it 

is  needed.  And  secondly,  even  if  the  training  is  remembered,  it  may  be¬ 
come  obsolete  before  it  can  be  employed. 

19-  What  the  above  discussion  suggests  is  the  possibility  of 

providing  new  recruits  with  a  basic  course  in  electronic  principles 
which  gives  them  only  what  they  are  reasonably  certain  to  need  in 
their  first  few  years  on  the  job.  Then,  if  they  re-engage,  more 
advanced  training  in  electronic  principles,  as  well  as  other 
technical  aspects  of  their  trade,  can  be  given  to  prepare  them  for 
the  next  several  years  of  their  careers. 
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20.  Such  a  scheme  is  certainly  not  appropriate  for  every  trade. 

However,  electronic  principles  survey  data  like  those  presented  in 
Table  3  can  be  used  to  determine  if  sufficient  numbers  of  tasks  in 
areas  related  to  one  another  fit  the  pattern  illustrated  (that  is, 
small  percentages  of  new  recruits  performing  tasks  and  using 
principles  to  larger  percentages  of  more  senior  people  performing 
tasks  and  using  principles)  to  warrant  a  post-graduate  type  of 
course.  In  the  first  electronic  principles  survey  conducted  in 
the  RAAF,  rt?spondents  from  all  six  technical  trades  surveyed 
indicated  that  some  electronic  principles  training  should  be  delayed 
for  inclusion  in  a  post-graduate  course.  The  question  was  not  posed 
in  tlu  'econd  electronic  principles  study. 


Principle/Task 

IHB[ 

Use  or  refer  to  pin  insertion  or 
extraction  tools 

30 

52 

55 

Measure  power 

25 

37 

41 

Use  knowledge  of  floating  reference 
potential 

27 

44 

46 

Adjust  transformers 

8 

20 

33 

Fault  find  synchro  systems 

38 

62 

61 

Operate  a  valve  tester 

33 

36 

53 

Operate  a  frequency  meter 

17 

33 

43 

Use  a  knowledge  of  the  ratings 
of  circuit  breakers 

27 

48 

54 

Use  the  term  conventional  current 
flows 

13 

36 

49 

Use  a  knowledge  of  the  generation 
of  voltage  by  chemical  action 

22 

41 

51 

Table  3*  Examples  of  Selected  Principles  Where  Training  May 
Be  Delayed  in  the  INSTFITT  Mustering 


Provided  Cost-Effective  Training 

21 .  The  philosophy  behind  providing  cost-effective  training 
has  already  been  briefly  discussed.  To  reiterate,  this  philosophy 
states  that  training  dollars  should  be  concentrated  on  those  skills 
and  knowledges  that  trainees  are  most  likely  to  make  use  of  in  the 
period  immediately  following  completion  of  their  training.  The 
length  of  that  period  will  be  a  function  of  several  factors  such 

as  how  technical  the  trade  is,  but  probably  shouldn't  extend  beyond 
the  first  enlistment  period. 

22.  Data  from  electronic  principles  surveys  provide  information 
concerning  those  electronics  skills  and  knowledges  used  by  airmen  in 
various  groups.  Those  data  can  be  compared  with  current  training 
syllabuses  to  determine  areas  of  overtraining  or  misdirected  train¬ 
ing.  Once  such  a  determination  is  made,  specific  adjustments  to 
existing  training  programmes  can  be  made,  so  that  electronic 
principles  actually  used  by  airmen  in  the  performance  of  their  jobs 
are  included  in  training  programmes  of  the  future,  and  so  that  "nice 
to  know"  but  little  used  information  is  eliminated  or  postponed  for 
inclusion  in  a  post-graduate  course. 


23.  Table  4  provides  an  example  of  how  survey  data  can  be  used 

to  identify  areas  of  possible  overtraining.  At  the  time  of  the 
first  electronic  principles  survey,  airmen  in  the  Armament  Fitter 
mustering  were  receiving  15  units  of  training  on  inductors,  and 
yet, as  can  be  seen  in  the  table,  only  very  small  percentages  of 
airmen  in  the  lower  ranks  were  using  these  principles  or  performing 
tasks  associated  with  inductors. 


Principle/Task  fo  Using 


I 

GENERAL  AREA:  INDUCTORS,  TIME  SPENT  15  UNITS 

Test  inductors  (coils)  6 

Fault  find  in  circuits  containing  inductors  (coils)  7 

Remove  or  replace  inductors  (coils)  6 

Adjust  inductors  (coils)  3 

Induced  voltages  in  a  coil  6 

Self  inductance  in  a  coil  6 

Back  emf  or  self  induced  voltage  in  a  coil  6 

The  unit  of  self  or  mutual  inductance  -  the  Henry  6 

The  factors  affecting  self  inductance  of  a  coil 
(eg  coil  size,  number  of  turns,  etc)  6 

The  method  of  finding  the  total  inductance  of  a  number 
of  inductors  in  series  and  parallel  5 

The  types  of  fixed  inductors  (iron  core,  iron  dust  core, 
air  core)  6 

The  uses  of  inductors  to  store  energy  or  oppose  current 
changes  5 

The  common  faults  of  inductors  (eg  shorted  turns,  open 
circuits,  etc)  6 

The  phase  relationship  between  applied  voltage  and 
circuit  current  for  a  purely  inductive  AC  circuit  5 

The  relation  Pav  =  0  for  a  purely  inductive  AC 
circuit  3 

The  graph  of  X-^  against  frequency  2 

Mutual  inductance  (M)  3 

Voltage  rating  of  inductors  4 

Frequency  ratings  of  inductors  3 

Current  rating  of  inductors  4 


Table  4.  Example  of  Possible  Overtraining  in  the 
ARMFITT  Mustering 

24.  While  an  example  is  not  presented  here,  it  is  easy  to 

envision  a  set  of  circumstances  where  little  or  no  training  is 
being  provided  in  an  area  in  which  airmen  are  called  upon  to  be 
proficient  when  they  reach  their  jobs  in  the  field.  Identifica¬ 
tion  of  both  areas  of  overtraining  and  undertaining  is  essential 
if  training  is  to  be  both  cost-effective  and  at  the  same  time 
effective  in  producing  graduates  who  are  able  to  perform  on  the  job 
with  little  additional  indoctrination.  The  EPI  is  one  tool  which 
can  help  in  the  identification  of  these  potential  problem  areas. 


SOME  CAUTIONS  IN  THE  APPLICATION  OF  SURVEY  DATA 


25.  It  should  be  obvious  from  the  above  discussion  that  the  EPI 

can  produce  data  which  have  broad  potential  utility  in  the  training 
environment.  At  this  point,  it  seems  appropriate  to  introduce  a  few 
cautions  which  must  be  heeded  in  the  application  of  those  data. 

Data  as  a  Tool 


26.  To  begin,  electronic  principles  survey  data  must  not  be  viewed 
as  a  panacea  -  they  do  not  provide  the  answer  to  all  our  electronics 
training  problems.  Rather,  they  are  more  appropriately  viewed  as  a 
tool  (and  hopefully,  a  very  effective  tool)  which  can  be  used  by  experts 
to  assist  them  in  applying  their  professional  skills  and  experience 

in  making  training  decisions.  Too  often,  users  of  survey  data  have 
taken  the  data  at  face  value  and  have  attempted  to  apply  them  without 
regard  for  extenuating" circumstances  or  special  cases.  Often  the 
results  are  of  much  less  utility  than  anticipated,  and  the  survey 
technique  is  blamed  for  being  inadequate  or  invalid.  The  numbers  do 
"speak  for  themselves",  but  they  need  someone  to  talk  to  -  someone 
who  will  apply  his  or  her  professional  judgement  in  the  interpretation 
and  application  of  the  data. 

Using  Data  In  Context 

27.  One  of  the  most  dangerous  pitfalls  in  attempting  to  apply  survey 
data  is  the  tendency  to  take  survey  results  out  of  context.  In  the 
example  cited  previously  on  the  use  of  inductors  by  members  of  the 
Armament  Fitter  mustering,  it  is  true  that  very  few  Armament  Fitters 

use  inductors  or  directly  employ  any  knowledge  of  inductors.  However, 
before  all  training  on  inductors  is  scrapped,  the  subject  matter 
specialist  must  be  certain  that  knowledge  of  inductors  is  not  essential 
to  the  understanding  of  other  principles  and/or  tasks  which  are  used 
or  performed.  Once  the  specialist  is  convinced  that  knowledge  of  the 
principles  of  inductors  is  not  used  directly  or  indirectly,  then 
appropriate  steps  can  be  taken  to  reduce  or  eliminate  training  on 
inductors.  The  important  point  is  that  portions  of  survey  data  must 
be  evaluated  within  the  context  of  all  the  survey  data;  indeed,  within 
the  context  of  the  mustering  and  its  activities  as  a  whole. 

Survey  Data  and  Policy 

28.  Throughout  this  paper,  it  has  been  necessary  to  make  one  under¬ 
lying  assumption  when  speaking  about  the  uses  of  survey  data  in  the 
training  environment.  The  EPI  determines  what  is  going  on  in  the  field. 
The  examples  presented  in  this  paper  for  the  use  of  electronic  principles 
survey  data  have  been  based  on  the  assumption  that  what  is  going  on 

in  the  field  is  what  ought  to  be  going  on  in  the  field.  Unfortunately, 
this  assumption  is  not  valid  in  every  instance. 


There  may  be  areas  within  a  survey  where  data  (what  is)  and  management's 
policy  (what  ought  to  be)  are  in  disagreement.  And  so,  before  survey 
data  are  applied  in  any  way,  those  responsible  for  making  and 
evaluating  policy  should  examine  the  data  to  determine  if  policy  is 
being  followed.  In  those  instances  where  policy  and  data  agree, 
application  of  the  data  to  training  problems  is  warranted;  however, 
in  those  areas  where  there  is  disagreement  between  policy  and  data, 
corrective  action  of  one  sort  or  another  will  be  necessary.  Either 
the  policy  is  wrong,  in  which  case  it  should  be  changed  and  survey  data 
can  be  applied;  or  policy  is  validated,  and  the  survey  data  pertaining 
to  those  areas  is  of  no  further  use,  having  served  the  purpose  of 
identifying  areas  where  policy  has  not  been  enforced.  This  procedure 
of  enlisting  the  aid  of  policy  makers  early  in  the  training  validation 
process  is  essential  if  we  are  to  avoid  perpetuating  inadequacies  in 
either  policy  or  performance,  or  both.  And  because  the  policy  makers 
are  involved  from  the  start  of  the  training  validation  process,  they 
are  much  more  likely  to  be  satisfied  with  the  resulting  training 
programme  that  emerges. 


SUMMARY 


29.  The  EPI  is  a  survey  instrument  which  differs  from  other 

occupational  survey  instruments  in  that  it  obtains  information  about 
knowledges  employed  on  the  job  as  well  as  tasks  performed;  and  it  can 
be  administered  to  any  group  of  people  who  work  with  electronics,  not 
just  to  members  of  one  trade.  It  has  been  developed  in  a  period  of 
rising  training  costs  and  increasingly  scarce  training  dollars  in  an 
effort  to  contribute  to  the  development  of  cost  effective  electronics 
training.  When  employed  as  a  tool  by  professionals  and  technical  spec¬ 
ialists  who  temper  survey  data  with  their  own  skill  and  experience, 
training  programmes  that  are  both  economical  and  motivating  can  result. 
Those  training  programmes  in  turn  will  produce  the  very  commodity  for 
which  they  exist:  motivated  personnel,  adequately  trained  at  minimum 
cost  to  perform  the  tasks  and  employ  the  principles  they  will  be 
called  upon  to  perform  and  employ  in  their  daily  jobs. 


I 


-  41  - 

CONFIDENTIALITY  OF  DATA  IN  OCCUPATIONAL  ANALYSIS  SURVEYS 


Major  A.  Davidson,  MA  (Ed  &  Psych) 
Military  Employments  Research  Information  Team 


INTRODUCTION 


1 .  Once  having  launched  into  this  topic  I  discovered  that  the 

actual  heading  of  this  paper  should  not  simply  have  been  "Confident 
iality  of  Data  in  Occupational  Analysis  Surveys".  The  other  side 

of  the  coin,  and  very  much  allied  with  confidentiality  is  the 
question  of  respondent  anonymity  or  lack  there  of.  If  total 
anonymity  is  to  be  decided  upon,  then  confidentiality  would  follow 
as  a  matter  of  course. 


2.  The  carrying  out  of  surveys,  be  they  for  the  purpose  of 

gathering  occupational  analysis  data  or  for  any  other  purpose, 
usually  entails  the  gathering  of  data,  some  of  which  at  least, 
respondents  would  not  like  advertised  or  to  have  themselves 
identified  with. 


3.  As  a  general  rule,  in  socio-metric  surveys,  where  random 

samples  are  used  and  where  follow-up  or  identification  is 
unnecessary,  total  anonymity  of  responses  is  accepted  as  a  standard 
operating  procedure. 


CONFLICT 


4.  However,  when  we  turn  to  occupational  analysis  surveys 
within  the  Services,  other  factors  begin  to  arise.  Firstly  the 
surveys  are  usually  aimed  at  1 00$  of  any  group  of  respondents. 
Secondly,  much  more  personal  information  is  sought.  For  a  variety 
of  reasons,  accurate  information  is  needed  on  ranks,  units,  time 

in  service  and  so  on.  Also  the  possibility  of  follow  up  studies 
can  exist  in  specific  trades  or  units. 

5.  The  question  (and  conflict)  thus  arises:  do  we  in  fact 
lose  reliable  data  when  we  insist  on  having  each  respondent 
identify  himself  to  the  extent  that  we  are  able  to  trace  the 
completed  booklet  directly  to  its  source?  Will  the  respondent, 
knowing  that  he  can  be  identified,  give  us  answers  which  he  thinks 
we  will  be  happier  with?  Or,  for  that  very  same  reason,  will  he 
give  us  more  honest  answers?  I  am  speaking  here,  mainly  about  the 
section  of  the  questionnaire  booklet  which  asks  personal  questions 
and  opinions,  such  as  Job  Satisfaction  and  Intention  to  Re-engage, 
rather  than  that  section  which  deals  strictly  with  the  tasks 
performed. 


TOTAL  ANONYMITY 


6.  We  in  the  Army  have  experienced  at  least  one  survey,  and 
a  reasonably  large  one  at  that,  where  total  anonymity  has  led  to 
an  unacceptably  large  percentage  of  incorrect  information  and 
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childish  filling  out  of  answer  sheets.  It  is  possible  however 
that  other  factors  (such  as  the  possibility  of  insufficient 
sponsor  corps  backing  or  interest)  may  have  contributed. 


TOTAL  IDENTIFICATION 


7.  The  opposite  approach  to  total  anonymity  is  to  ask  for 
all  identifying  information  to  be  provided.  Number,  rank,  name 
and  unit,  and,  I  believe  with  the  United  States  Air  Force,  even  a 
contact  telephone  number  is  requested.  As  I  understand,  this 
approach  results  in  acceptable  response  rates. 

8.  When  a  specific  question  arises  concerning  a  group  of  tasks 
and  members  of  a  trade,  a  direct  telephone  call  may  be  made  to  clear 
up  any  confusion.  This  I  believe  is  also  a  method  of  publicising 
the  work  done,  and  existence  of,  the  Occupational  Analysis  network. 


ARMY  APPROACH 


9.  We  in  the  Army  have  steered  a  somewhat  middle  path.  For 
identification  we  ask  unit  and  Army  Number  plus  rank.  In  addition 
we  strongly  stress  both  in  introductory  letters  and  the  instruction 
letter  accompanying  the  survey  booklet,  that  the  respondents  answers 
will  be  seen  by  no-one  outside  the  occupational  analysis  Section. 

10.  Our  main  problem  as  I  see  it,  is  that  this  confidentiality 
must  not  only  be  observed  in  practice,  but  must  be  seen  to  be 
observed  by  the  respondent.  To  have  the  (say)  Chief  Clerk,  or 
Adjutant  of  any  unit  getting  together  and  having  a  quiet  chuckle 
over  some  respondents  efforts,  is  a  certain  way  to  destroy  credib¬ 
ility  if  it  becomes  common  knowledge  among  soldiers. 

11.  We  have  also  had  experience  of  respondents'  supervisors 
actually  amending  the  completed  booklets  in  the  honest  belief  that 
they  were  doing  the  right  thing. 


WHAT'S  THE  ANSWER 


12.  Having  at  one  time  or  another  been  involved  with  surveys  of 
almost  every  type,  it  would  appear  to  me  that  the  basic  goal  we 
must  strive  for  is  not  so  much  deciding  the  scope  of  personal 
information  we  seek,  but  to  ensure  that  the  confidential  treatment 
of  that  information  is  made  as  effective  as  possible.  This 
confidential  treatment  must  be  made  obvious  to  the  respondent.  He 

must  be  able  to  see  that  if  he  does  answer  honestly  and  so  perhaps 
in  a  manner  unacceptable  to  his  superiors,  his  own  personal  career 
is  totally  protected.  If  we  get  the  soldier  to  trust  us,  he  will 
give  us  information  that  we  can  trust. 
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13.  Having  such  a  variety  of  units  in  such  diverse  locations, 
we  are  not  in  a  position  to  use  methods  so  successfully  employed 
by  the  Air  Force.  The  Navy  has  two  main  methods  of  survey 
administration.  These  are  somewhat  similar  to  our  own.  The 
first  method  is  where  members  of  the  Navy  Occupation  analysis  cell 
actually  go  out  to  areas  of  concentration  of  members  of  the  trade 
involved.  They  distribute,  administer,  and  collect  the  booklets. 
The  second  method  is  to  send  bundles  of  booklets  to  contact  officers 
in  inaccessible  areas  such  as  ships  at  sea,  or  remote  areas  such  as 
Darwin.  In  the  areas  where  the  Navy  has  training  research  units, 
these  are  used  as  contact  points.  The  Air  Force  on  the  other 

hand  uses  Presidents  of  Unit  Trade  Testing  Boards.  These  personnel 
are  responsible  for  questionnaire  administration  at  unit  level. 

The  majority  of  Air  Force  units  have  these  trade  test  boards.  In 
the  few  locations  where  these  boards  do  not  exist,  unit  commanding 
officers  are  responsible.  This  system  has  the  great  advantage  of 
insuring  that  surveys  are  administered  by  trained  personnel  under 
control  conditions.  We  in  the  Army  are  forced  to  use  local  dis¬ 
tributing  and  briefing  officers,  many  of  whom  have  the  task  thrust 
upon  them  perhaps  unwillingly. 

14.  Instructions  on  questionnaire  explanation  are  normally  set 
out  in  a  letter  which  is  never  as  successful  as  the  personal 
approach.  We  should  aim  at  lessening  the  amount  of  post  survey 
work  for  these  people.  So  as  I  asked  before,  What  is  the  answer? 

It  would  appear  then  that  in  the  long  run  our  only  answer  could 
possibly  be  the  use  of  individual  return  addressed  envelopes,  or 
in  view  of  comments  made  thus  far  by  Dr  Christal,  perhaps  this 
problem  is  not  in  fact  important. 
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OCCUPATIONAL  ANALYSIS  OF  RAN  OFFICERS 


Lieutenant  Commander  R.  Hawketts,  BSc,  RAN 
Staff  Officer  Occupational  Analysis 


1 .  Occupational  Analysis  work  in  the  Australian  Services 

generally,  and  the  Navy  in  particular,  is  relatively  new  and 
consequently  its  methodology  is  heavily  biassed  towards 
published  research  and  survey  reports  from  overseas.  As  in  all 
such  work,  the  best  and  most  obvious  return  for  effort  is  made  in 
studies  covering  large  population  groups,  so  it  is  hardly  surpris¬ 
ing  that  most  of  this  published  work  refers  to  surveys  of  "other 
rank"  employments.  Nevertheless  the  Officer  Corps,  although 
forming  less  than  fifteen  percent  of  the  Navy's  strength  is  vitally 
important  to  its  efficient  functioning  and  I  am  certain  that  this 
applies  equally  well  to  any  other  Military  Service.  This  was 
clearly  appreciated  when  the  Navy's  Occupational  Analysis  Unit  was 
tasked  by  the  Chief  of  Naval  Personnel  with  a  survey  to  gather 
information  useful  in  monitoring  what  was  termed  "the  Officer 
development  process".  The  integral  parts  of  this  process  are 
taken  to  be  formal  and  Service  education,  training  and  career 
planning.  The  expression  itself  and  its  definition  closely 
matches  that  used  by  the  Army  in  its  recent  Regular  Officer  Develop¬ 
ment  Committee  investigations.  Obviously  then,  the  Officer 
development  process  is  closely  related  to  the  career  pyramid,  and 
its  factors  run  through  rank  structure. 


DIAGRAM  1 


I 

I 


EDUCATION 
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2.  Diagram  1  shows  the  generally  accepted  ideal  career 

pyramid  with  the  factors  of  the  Officer  development  process 
starting  at  the  base  with  the  lowest  ranks  and  (hopefully) 
running  through  to  the  highest.  If  an  Officer's  aims  and  desires 
are  not  met  by  his  particular  development  plan,  then  there  is  a 
danger  of  him  being  lost  to  the  Service  with  consequent  changes 
in  the  pyramid  shape  (Diagram  2).  The  stages  at  which  these 
changes  occur  will  depend  on  whether  this  mismatch  is  related  to 
rank,  branch  or  domestic  circumstances. 


DIAGRAM  2 


The  shortfall  in  numbers  by  ranks  is  indicated  by  the  horizontal 
distances  A  and  B  in  diagram  2. 

3.  The  problem  of  postings  is  compounded  by  promotion 

difficulties  at  the  higher  ranks  as  the  latitude  for  selection 
diminishes.  This  may  in  turn  affect  the  quality  of  the  upper 
echelon  since  it  would  be  unwise  to  assume  that  only  those  who 
would  have  been  promoted  in  any  case,  remained  in  the  Service. 

To  regain  the  optimum  pyramid  shape,  the  base  can  be  broadened 
with  consequent  increase  in  training  and  recruitment  costs,  and 
perhaps  later  different  problems  if  anticipated  wastage  rates  are 
not  met.  A  further  problem  here  may  be  that  the  entry  quality 
standards  may  have  to  be  reduced  to  meet  the  quantitative  needs. 
The  alternative  is  to  attempt  to  cut  back  on  losses  by  changes  in 
the  development  process  itself.  These  changes  would  be  the 
result  of  Service  considerations  supported  or  amended  as  necessary 
by  expressed  opinions,  attitudes  and  aims  of  the  individuals  being 
developed. 


•PMMJIII1* 
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4.  The  second  is  clearly  the  more  sensible  course  and  the 
Navy's  Survey  of  Officers  is  using  a  questionnaire  designed  to 
gather  opinion  on  the  range  of  Officer  Development  Process  Factors, 
for  ranks  up  to  Commander. 

5.  Table  1  shows  the  seven  sections  of  the  questionnaire,  and 
with  the  exception  of  the  first,  all  call  for  estimates  or  assess¬ 
ments  from  the  job  incumbent. 


TABLE  1 


RAN  OFFICER  SURVEY 


1 .  Personal  Details 

2.  Qualities  and  Skills 

3.  Opinions  of  Rank  and  Branch 

4.  Job  Responsibilities 

5.  Career  Satisfaction 

6 .  Management  Organisation 

7.  Educational  Needs 

6.  Coarse  job  appraisals  are  called  for  in  Sections  2,  3  and 

6.  using  different  parameters: 

a.  Abilities  -  based  on  the  Officer  assessment  form. 

b.  Management  -  using  the  factors  defined  in  the  RAN 

Duty  Statement  form. 

c.  Rank  Opinion 

Hopefully  there  will  be  an  acceptable  consistency  of  estimates 
within  the  one  job,  to  allow  ranking  of  jobs  in  terms  of  one  or 
more  of  the  factors  used.  It  is  anticipated  that  this  classifica¬ 
tion  of  jobs  will  allow  better  matching  of  men  and  jobs  and, 
through  identification  of  jobs  without  branch  or  specialisation 
requirements,  will  give  some  flexibility  in  postings. 

7.  A  finer  breakdown  of  the  job  content  is  given  in  Section  4, 
although  this  by  no  means  approaches  the  level  of  task  detail  used 
in  "other  rank"  surveys.  The  assessments  of  adequacy  of  training 
given  for  the  activities  listed,  should  provide  some  guidance  to 
trainers,  and  will  at  least  identify  training  areas  and  activities 
to  be  subjected  to  more  detailed  study. 

8.  As  I  said  earlier,  the  ability  to  plan  careers  is  reduced 
as  wastage  increases  to  the  stage  where  the  available  manpower 
leaves  no  margin  for  selection  in  either  postings  or  promotions. 

It  is  possible  that  wastage  is  equated  to  a  range  of  Service 
conditions,  financial  and  others,  such  as  educational  or  travel 
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4.  The  second  is  clearly  the  more  sensible  course  and  the 
Navy's  Survey  of  Officers  is  using  a  questionnaire  designed  to 
gather  opinion  on  the  range  of  Officer  Development  Process  Factors, 
for  ranks  up  to  Commander. 

5.  Listed  hereunder  are  the  seven  sections  of  the  questionnaire: 

a.  Section  1  -  Personal  Details. 

b.  Section  2  -  Qualities  and  Skills. 

c.  Section  3  -  Opinions  of  Rank  and  Branch, 
u.  Section  4  -  Job  Responsibilities. 

e.  Section  5  -  Career  Satisfaction. 

f.  Section  6  -  Management  Organisation. 

g.  Section  7  -  Educational  Needs. 

With  the  exception  of  the  first,  all  sections  call  for  estimates 
or  assessments  from  the  job  encumbent. 

6.  Coarse  job  appraisals  are  called  for  in  Sections  2,  3  and 

6.  using  different  parameters: 

a.  Abilities  -  based  on  the  Officer  assessment  form. 

b.  Management  -  using  the  factors  defined  in  the  RAN 

Duty  Statement  form. 

c .  Rank  Opinion 

Hopefully  there  will  be  an  acceptable  consistency  of  estimates 
within  the  one  job,  to  allow  ranking  of  jobs  in  terms  of  one  or 
more  of  the  factors  used.  It  is  anticipated  that  this  classifica¬ 
tion  of  jobs  will  allow  better  matching  of  men  and  jobs  and, 
through  identification  of  jobs  without  branch  or  specialisation 
requirements,  will  give  some  flexibility  in  postings. 

7.  A  finer  breakdown  of  the  job  content  is  given  in  Section 
4,  although  this  by  no  means  approaches  the  level  of  task  detail 
used  in  "other  rank"  surveys.  The  assessment  of  adequacy  of  train¬ 
ing  given  for  the  activities  listed,  should  provide  some  guidance  to 
trainers,  and  will  at  least  identify  training  areas  and  activities 
to  be  subjected  to  more  detailed  study. 

8.  As  I  said  earlier,  the  ability  to  plan  careers  is  reduced 
as  wastages  increases  to  the  stage  where  the  available  manpower 
leaves  no  margin  for  selection  in  either  postings  or  promotions. 

It  is  possible  that  wastage  is  equated  to  a  range  of  Service 
conditions,  financial  and  others,  such  as  educational  or  travel 
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opportunities.  Section  5  will  give  a  feel  for  the  relative 
degrees  of  satisfaction  with  a  wide  range  of  Service  conditions 
and  permit  comparison  of  satisfaction  levels  between  branches, 
ranks  and  specialisations.  The  final  section  will  measure  the 
requirement  for  education  subjects  within  jobs. 

9.  For  this  survey  over  1900  questionnaires  have  been  issued, 
one  to  each  established  billet.  At  this  stage  62$  of  the  booklets 
issued  have  been  returned,  and  we  shall  establish  the  first 
computer  file  within  the  next  two  weeks.  The  broad  intent  is  to 
provide  directors  initially  with  basic  outputs  that  are  considered 
relevant  to  their  responsibilities,  and  then  to  invite  direction 
with  respect  to  more  precise  information  needs. 

10.  A  large  number  (35$)  of  the  respondents  have  taken  the 
opportunity  to  pass  to  Navy  Office  their  comments  on  various 
aspects  of  Service  life  and  conditions  which  they  felt  the 
questionnaire  did  not  investigate  in  sufficient  depth.  These 
also  are  being  analysed  and  any  further  questionnaires  will  be 
improved  in  the  light  of  much  constructive  comment. 
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OFFICER  JOB  ANALYSIS  -  ARMY 


Lieutenant  Colonel  M.J.  Eley,  BA,  Dip.  Ed. 
Deputy  Director  of  Psychology  -  Army 


1 .  I  want  briefly  to  describe  two  surveys  that  the  Military 
Employments  Research  Information  Team  (MERIT)  carried  out  for  the 
Regular  Officer  Development  Committee  (RODC)  in  1977.  Both 
surveys  were  conducted  on  samples  of  Army  Officers  and  both  were  in 
response  to  specific  requests  from  that  committee  for  objectively 
derived  information  on  Army  Officers. 

2.  I  have  subtitled  my  presentation  "The  Importance  lies  in 
the  Grain  of  Salt",  because  although  I  believe  both  surveys 
answered  most  of  the  questions  posed,  I  also  believe  the  inform¬ 
ation  and  its  interpretation  is  subject  to  specific  limitations, 
mainly  because  the  surveys  represent  an  attempt  to  apply  very 
objective  measures  to  very  unstructured  jobs,  and  employment  groups. 
I  hope  that  Dr  Christal  during  his  visit  will  be  able  to  comment 

on  some  of  the  doubts  I  will  express. 

3.  The  first  survey  was  in  response  to  a  question  from  the 
RODC  which  wanted  to  know  the  opinions  of  officers  on  the  relevance 
of  areas  of  education  to  appointments,  ranks,  and  career  develop¬ 
ment.  We  decided  to  design  a  survey  that  almost  replicated  one 
conducted  by  Morsh  on  USAF  Officers  in  1969. 

4.  A  list  of  122  education  topics  that  might  normally  not  be 
the  subject  of  formal  course  training,  was  developed  by  a  panel  of 
selected  officers  from  a  wide  range  of  backgrounds  and  experience, 
using  Morsh' s  list  as  a  basis.  The  topics  were  grouped  as  shown  in 
Table  1 . 


TABLE  1 

TOPIC  HEALINGS 

1 .  National  Security  Policy  and  Strategy 

2.  International  relations  and  Economics 

3.  Organisation  of  Military  Forces 

4.  Military  Operations 

5.  Military  Technology 

6 .  Military  History 

7.  Joint  and  Allied  Warfare 

8.  Logistics 

9.  Management  Skills  (Individual) 

10.  Management  Skills  (General) 

11.  Management  Theory  (Leadership  and 
Human  Relations) 

12.  Management  Theory  (Organisations  and 
Systems) 


(Containing  15  Topics) 
(22  Topics) 

(9  Topics) 

(11  Topics) 

(3  Topics) 

(2  Topics) 

(5  Topics) 

(8  Topics) 

(10  Topics) 

(7  Topics) 

(6  Topics) 

(9  Topics) 


13.  Training 

14.  General  Topics 


(5  Topics) 
(8  Topics) 


5.  The  122  topics  were  included  in  a  questionaire,  where  for 
each  topic,  two  questions  were  asked: 

a.  Need  to  Know  (NTK) 

"Rate  the  extent  to  which  you  consider  an  Army 
Officer  in  general  needs  knowledge  of  this  topic 
for  effective  development  in  his  or  her  career". 

(The  NTK  question  was  designed  to  measure  the 
degree  to  which  officers  saw  a  general  need  for 
knowledge  in  that  area.  A  second  question  was 
asked  in  an  attempt  to  measure  a  more  realistic 
job  related  to  need). 

b.  Need  on  Job  (NOJ) 

"Rate  the  extent  to  which  you  consider  an  officer 
of  your  rank  in  your  present  job  actually  needs 
knowledge  of  this  topic  to  perform  his  or  her  duties" 

6.  Using  CODAP  a  variable  representing  the  average  numerical 
response  to  the  NTK  at  NOJ  questions  for  each  topic  was  calcu¬ 
lated,  for  difference  groups  selected  from  the  sample.  The  size  of 
this  "average"  variable  indicated  the  extent  to  which  that  group 

of  officers  recognised  a  need  to  know  that  topic,  in  general,  or  a 
need  in  their  own  .job. 

7.  Because  of  time  I  can’t  go  into  detail  about  the  results, 
but  byway  of  example  I  shall  show  a  couple  of  summaries.  Firstly, 
taking  NOJ  as  the  most  significant  measure  of  actual  need,  the 
following  ten  topics  were  identified  as  most  needed  by  the  sample 
as  a  whole: 

1  .  Written  Communication  Skills 

2.  Oral  Communication  Skills 

3.  Leadership  and  Man  Management 

4.  Discipline  and  Morale 

5.  Security  of  Classified  Matter 

6.  Techniques  of  Logical  Thinking 

7.  Organisation  and  Role  of  Functional  Commands  and 
Military  Districts 

8.  Role  and  Employment  of  the  Australian  Army 

9.  Problem  Solving  Techniques 


10.  Counselling  and  Evaluating 
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It  is  perhaps  no  coincidence  that  the  bulk  of  these  topics  come 
from  the  area  of  Individual  Management  Skills.  This  fitted  into 
what  the  RODC  had  been  finding  out  from  other  areas. 

8.  Another  example,  again  using  mean  NOJ,  compares  the  "most 

needed"  topics  rated  by  various  ranks: 

Second  Lieutenant 

1 .  Safety 

2.  Leadership  and  Man  Management 

3.  Discipline  and  Morale 

4.  How  the  Army  Logistic  System  Works 

5.  Written  Communication  Skills 

6.  Unit  Organisation,  Tactics  etc 

7.  Counselling  and  Evaluating 

8.  Oral  Communication  Skills 

9.  Military  Law 

10.  Health  and  Hygiene 

Junior  officers,  because  of  the  way  in  which  they  are  employed, 
tend  to  rank  Management  Theory,  Training  and  Safety  etc,  highly. 

As  rank  increased,  Individual  Management  Skills  became  more 
important,  and  topics  from  areas  such  as  "Army  Organisation  and 
Role",  as  opposed  to  "Unit  Organisation  and  Tactics,  became  also 
more  significant. 

Ma.i  or 

1 .  Written  Communication  Skills 

2.  Oral  Communication  Skills 

3.  Conference  and  Committee  Techniques 

3.  Leadership  and  Man  Management 

4.  Techniques  of  Logical  Thinking 

5.  Organisation  and  Role  of  Functional  Commands  and 
Military  Districts 

6.  Role  and  Employment  of  Australian  Army 

7.  Problem  Solving  Techniques 

8.  Techniques  of  Creative  Thinking 

9.  How  the  Army  Logistic  System  Works 
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9.  The  RODC  in  fact  asked  for  printouts  showing  the  top  40 
odd  topics  for  each  rank,  and  they  were  looking  for,  and  found. 
Confirmation  for  other  findings  of  theirs  in  this  survey. 

10.  The  second  survey  was  an  officer  job  analysis,  a  more 
traditional  kind.  The  RODC  wanted  to  know,  the  duties  and  tasks 
associated  with  particular  ranks,  types  of  appointments  and 
corps  of  officers.  Also,  the  best  way  of  grouping  like  officer 
appointments . 

11.  For  the  officer  job  analysis,  Morsh's  ideas  again  were 
generally  followed,  but  some  significant  differences  were  incor¬ 
porated.  Again  using  careful  investigation,  and  scrutiny  by  a  wide 
range  of  Officers,  we  developed  two  separate  lists  of  tasks.  One 
was  a  list  of  management  functions,  and  these  tasks  came  under  the 
management  -  oriented  headings  of: 

Commanding  and  Directing, 

Supervising, 

Planning, 

Organising, 

Evaluating, 

Coordinating, 

Communicating,  and 

Implementing . 

This  was  the  structure  used  by  Morsh.  Our  other  task  list  was  a 
list  of  normal,  functional  tasks,  covered  by  topic  headings  such 
as : 

Operations, 

Drill  and  Ceremonial, 

Policy, 

Training, 

Discipline, 

Personnel  Employment, 

Personnel  Support, 

Logistics,  and 

Army  Works,  Movements  etc. 


.  ■  i*n  ~  1*1  *i  rf  * ■>»*»*<> 
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12.  These  are  the  normal  functional  groups  we  associate  with 
occupational  analysis.  But  the  task  inventory  didn’t  contain  all 
tasks  performed  by  all  officers.  It  contained  300  odd  tasks 
selected  because  they  were  known,  or  thought,  to  be  common  to  many 
jobs.  Because  most  of  the  tasks  performed  by  an  officer,  or  for 
that  matter  by  anyone  removed  from  the  strictly  controlled  man- 
machine  or  routine  environment  are  difficult  to  measure  accurate¬ 
ly  by  the  normal  means,  which  involves  assessing  the  relative  time 
spent  on  each  task,  we  used  a  measurement  scale  that  we  believed 
suited  officer  type  tasks.  For  each  task  that  an  officer  perform¬ 
ed  he  responded  to  this  question: 

"For  each  task  which  you  recognise  as  a  part  of  your  job,  rate 
its  contribution  to  your  job  tin  peace  or  war)  on  the  following 
scale : " 

1 .  A  minor  Part  of  my  Job 

2. 

3. 

4. 

5.  A  Substantial  Part  of  My  Job 

6. 

7. 

8. 

9.  A  most  significant  Part  of  My  Job 

From  this  scale,  a  variable  called  Average  Percent  Contribution  to 
Job  (APCJ)  was  extracted  for  each  task  and  this  was  the  variable 
which  was  useJ  most  effectively  to  describe  and  compare  the  jobs 
of  officers  by  rank,  type  of  employment  and  other  categories. 

13.  But  we  used  two  task  inventories,  and  because  each  task 
listing  in  itself  described  the  same  range  of  activities  from  a 
different  viewpoint  it  was  obvious  that  to  combine  the  lists  when 
they  were  analysed  would  be  to  mix  up  and  duplicate  overlapping 
tasks.  So  in  effect  two  separate  surveys  were  conducted  together. 
The  different  task  lists  were  analysed  separately,  and  inter¬ 
preted  separately,  and  interpreted  separately  at  first.  For  the 
RODC,  this  was  sufficient  but  for  a  proper  analysis  of  officer 
jobs  these  two  series  of  results  were  able  to  be  analysed  as 
complementary  ways  of  describing  officers  jobs. 

14.  A  couple  of  the  results  we  obtained  can  be  illustrated 
by  the  table  at  Annex  A.  The  measure  of  the  involvement  in  each 
duty  by  each  rank  level  is  average  percent  contribution  to  job 

( APCJ ) .  The  survey  showed  that : 


i 
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a.  APCJ  declines  when  rank  increases,  for,  Operations 
and  Personnel  Support  duties; 

b.  APCJ  peaks  at  lieutenant  and  Captain,  for.  Training 
and  Discipline  duties; 

c.  APCJ  peaks  at  Captain  and  Major,  for.  Logistics  and 
Works,  Movement  duties,  and 

d.  APCJ  increases  and  rank  increases,  for,  Policy  and 
Personnel  Employment  duties. 

15.  This  normal  functional  task  inventory  was  used  to  extract 
quite  a  lot  of  information  about  the  employment  of  officers.  The 
CODAP  cluster  analysis  program  identified  thirty  two  identifiable 
groups  of  officer  groups,  thereby  providing  information  of  some 
value  to  the  RODC.  That  committee  had  also  asked  us  to  measure 
the  difference  in  the  employment  of  certain  groups  of  officers. 

Among  these  were  Officers  commissioned  from  senior  NCO  rank 
(Administrative  and  Technical  Officers),  and  women.  Annex  B 
compares  the  employment  of  these  two  groups  against  the  all  officer 
sample.  Predictably,  women  perform  less  in  the  areas  of  Policy, 
Training  and  Discipline,  and  more  in  the  areas  of  personnel 
employment  and  personnel  support.  Administrative  and  Technical 
Officers,  who  are  usually  employed  in  administrative  and  Q  work, 
predictably  do  more  in  the  areas  of  logistics,  works  and  movement, 
and  discipline,  as  well  as  following  the  women  in  Personnel  Empl 
ment  and  Personnel  Support.  At  the  task  level,  some  of  the  specific 
tasks  performed  more  by  women  than  men  were: 

a.  Personnel  Employment 

Interview  applicants,  interview/Counsell  and  provide 
personnel  information  on  request, 

b.  Personnel  Support. 

Promulgate  and  explain  Service  Conditions  policy 
and  process  applications  for  allowances, 

c.  Logistics. 

Conduct  a  stocktake  of  stores/equipment  and  amend 
entitlement  documents,  and 

d.  Miscellaneous. 

Account  for  Public  Monies. 

16.  Analysis  of  the  other  task  inventory,  the  one  categorised 
by  management  functions  rather  than  ordinary  specific  tasks, 
resulted  in  a  deal  of  information  about  who  does  what  in  the  area  of 
management.  One  example  is  at  Annex  C,  which  shows,  by  rank,  that: 


I 
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a.  Involvement  increases,  as  rank  increases,  in  the 
functions  of  Commanding  and  Directing,  Planning, 
Evaluating,  and  Communicating. 

b.  On  the  other  hand,  involvement  decreases  as  rank 
increases  in  the  functions  of  Supervising  and 
Implementing. 

This  showed  almost  the  same  tendency  as  that  obtained  by  Morsh 
eight  years  earlier  in  the  United  States. 


SUMMARY 


17.  A  great  deal  of  data  was  collected  and  used  in  those  surveys, 
and  the  results  made  sense,  supported  much  of  what  was  known,  and 
produced  some  new  information  about  officer  jobs  that  wasn’t 
previously  known.  It  was  the  first  across-the-board  comprehensive 
officer  job  analysis  conducted  in  this  country.  In  one  way  it  was 

an  effective  model  for  other  officer  surveys,  in  other  ways  it 
was  an  opportunity  for  ourselves  and  others  to  learn,  and  improve 
this  type  of  survey  in  future. 

18.  I  was  concerned,  however,  by  one  aspect  of  this  survey 
and  most  of  the  others  in  which  I  was  involved  while  I  was  with 
MERIT.  There  is  an  occasional  tendency  for  users  to  grasp  inform¬ 
ation  from  such  surveys,  and  believe  the  results,  literally,  and 
exclusively.  The  RODC  was  fairly  sensible  in  its  approach,  but 
others  who  had  access  to  the  data  tended  to  believe  that  because 
it  was  objectively  derived,  it  could  not  be  disputed.  I  would 
have  thought  that  it  was  commonsense  for  the  results  always  to  be 
applied  together  with  a  grain  of  salt.  It  is  a  responsibility  of 
those  involved  in  occupational  analysis  not  to  just  sing  the  praises 
of  their  technique,  but  to  advise  users  on  its  limitations,  so  that 
it  is  used  to  best  effect,  and  not  claimed  to  be  something  that  it 
is  not.  The  grain  of  salt  was  necessary  in  this  survey  for  three 
reasons : 

a.  The  tasks  were  not  representative  of  all  Officer 
tasks , 

b.  When  some  sub  groups  were  isolated  in  some  of  the 
smaller  Corps,  the  small  sample  size  made  the  results 
unreliable,  and, 

c.  Officers  in  the  Australian  Army  don't  specialize 
to  the  extent  that  their  US  counterparts  do;  their 
jobs  change  as  new  incumbents  arrive  and  depart,  and 
therefore  although  survey  information  is  known  to 

be  accurate  at  the  time  of  collection,  particularly 
with  an  officer  survey,  it  goes  out  of  date  very 
quickly,  and  consequently  management  decisions 
relying  on  survey  information  must  bear  this  in  mind. 
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19.  Having  raised  one  or  two  doubts  about  officer  job 

analysis  I  would  like,  in  conclusion,  to  address  two  questions  to 
Dr  Christal,  in  the  hope  that  the  state  of  the  art  has  advanced 
far  enough  in  the  last  two  years  for  him  either  to  be  able  to 
answer  them  directly,  or  else  discuss  them  during  this  Seminar. 

My  two  questions  are: 

a.  To  what  extent  are  officer  jobs  suited  to  objective 
job  analysis  techniques?  and  secondly,  and  more 
generally, 

b.  Using  occupational  analysis,  how  can  we  effectively 
measure  operational  involvement  in  a  peacetime 
Army? 


OFFICER  JOB  ANALYSIS 
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RAAF  OFFICER  EDUCATION  AND  TRAINING  SURVEY 


Wing  Commander  J.T.  Huet,  BSc,  BEd 


INTRODUCTION 


1 .  An  Officer  Education  and  Training  (OET)  Design  Team, 

comprising  Squadron  Leader  M.R.  Sykes  and  myself,  was  established 
at  Headquarters  Support  Command  on  1  March  1 979  to  review  general 
officer  education  and  training  in  the  RAAF.  As  a  starting 
point,  the  team  was  to  assume  the  OET  scheme  proposed  by  the 
1973/74  Air  Force  Office  Working  Party  on  officer  education  and 
training  requirements.  The  Working  Party’s  proposed  OET  scheme 
is  shown  at  Table  1 . 


TABLE  1 


PROPOSED  OFFICER  EDUCATION  AND  TRAINING 
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2.  The  recommendations  of  the  1973/74  Working  Party  have 
been  taken  as  starting  points  only  for  the  present  review  of 
OET.  The  Working  Party,  having  been  established  on  a  part-time 
basis,  admitted  in  its  report  that  its  findings  were  based  on  a 
subjective  analysis;  it  was  not  possible  in  the  time  available 
to  engage  in  research  of  an  objective  nature  to  establish  train¬ 
ing  objectives.  Thus,  there  were  areas  where  detailed  study 
needed  to  be  undertaken, 

3.  Because  of  the  lack  of  objective  data  on  which  to  base 
decisions  on  OET  requirements,  the  design  team's  first  task  was 
to  conduct  a  survey  of  RAAF  officers  to  determine  exactly  what 
RAAF  officers  do  and  what  general  Service  education  they  require. 
In  particular,  the  survey  was  to  be  used  to  answer  a  number  of 
the  team's  terms  of  reference  (Table  2).  The  main  ones  in 
relation  to  the  survey  are  terms  of  reference  1,  2,  3  (part)  4 
and  5o 


TABLE  2 


TERMS  OF  REFERENCE 


1 .  EVALUATE  THE  REVISED  OET  SCHEME  AS  PROPOSED  BY  THE  1 973/74 
AIR  FORCE  OFFICE  WORKING  PARTY  (IN  TERMS  OF  EFFECTIVENESS 
OF  THE  TRAINING  FOR  PREPARING  OFFICERS  FOR  FUTURE  APPOINT¬ 
MENTS).  FORMULATE  A  BROAD  AIM  FOR  THE  PROPOSED  OET 
SCHEME. 

2.  DETERMINE  THE  INTEGRATED  TRAINING  OBJECTIVES  (BOTH 
BEHAVIOURAL  AND  ATTITUDINAl)  FOR  THE  SEQUENCE  OF  COURSES. 

3.  DETERMINE  THE  BROAD  CONCEPT  OF  EACH  COURSE,  ITS  DURATION, 

THE  NUMBER  OF  STUDENTS  ON  EACH,  THE  APPROXIMATE  NUMBER  OF 
STAFF  AND  SUPPORTING  FACILITIES  AND  THE  ANNUAL  FREQUENCY 
OF  COURSES, 

4.  IDENTIFY  THE  CORE  SUBJECTS  FOR  GD  AND  NON-GD  STUDENTS  FOR 
INITIAL  OFFICER  TRAINING  AND  DETERMINE  WHERE  THESE  SUBJECTS 
WILL  BE  TAUGHT. 

5.  IDENTIFY  THE  NON-CORE  OET  SUBJECTS  FOR  BOTH  GD  AND  NON-GD 
STUDENTS,  AND  WHERE  THESE  SUBJECTS  WILL  BE  TAUGHT. 

6.  EVALUATE  THE  RODC  PROJECT  TO  DETERMINE  WHICH  OF  THE  FINDINGS 
ARE  APPLICABLE  TO  THE  RAAF  OET. 

7.  RECOMMEND  THE  MOST  SUITABLE  LOCATION  FOR  THE  OFFICERS' 
CORRESPONDENCE  COURSES  (PHASES  A  AND  B)  AND  THE  OFFICERS' 
STAFF  AND  MANAGEMENT  COURSE. 

8.  EXAMINE  THE  NEED  FOR  SPECIALIST  COURSES  TO  REPLACE  SPECIALIST 
PROMOTION  EXAMINATIONS  (IF  NECESSARY). 


9 


DETERMINE  THE  RESOURCES  REQUIRED  TO  IMPLEMENT  THE  STAGES  OF 
THE  REVISED  OET  PROGRAMME. 


10.  ESTIMATE  THE  COST  OF  IMPLEMENTING  THE  NEW  SCHEME. 

1 1 .  PREPARE  A  DRAFT  PLAN  FOR  THE  CHANGEOVER  FROM  THE  PRESENT  OET 
SCHEME  TO  THE  PROPOSED  SCHEME. 


SURVEY  ADMINISTRATION 


4.  To  obtain  the  information  required  of  officers,  the  team 

developed  a  survey  booklet.  The  booklet  consisted  of  three 
sections,  viz: 

a.  Background  Information.  Information  on  the 
background  of  each  respondent  was  collected  to 
allow  grouping  of  respondents  and  comparison 
of  these  groups.  Examples  of  background 
information  collected  are: 

( 1 )  Rank . 

(2)  Category. 

( 3 )  Current  Employment . 

(4)  Number  of  Immediate  Subordinates. 

(5)  Number  of  Persons  For  Whom  Responsible. 

(6)  Job  Title. 

(7)  Unit. 

The  use  of  these  factors  will  be  explained 
later. 

b.  Task  Inventory.  The  task  inventory  section 
contained  a  comprehensive  list  of  general 
administrative  and  managerial  tasks  that 
officers  are  likely  to  perform.  The  tasks 
were  listed  under  headings  which  could  be 
used  later  for  subject  headings  in  course 
syllabuses.  The  headings  were: 

(1)  Written  Communication. 

(2)  Oral  Communication. 

(3)  Drill  and  Ceremonial. 

(4)  General  Administration. 

(5)  Counter  Intelligence. 
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(6)  Personnel  Administration. 

(7)  Secondary  Duties. 

(8)  Planning. 

(9)  Organizing. 

(10)  Discipline. 

(11)  Training. 

d.  Educational  Topics  Inventory.  The  educational 
topics  inventory  contained  all  the  educational 
topics  currently  taught  to  officers  plus  many 
others  that  the  team  determined  may  be  required 
by  officers.  The  educational  topics  were  also 
listed  under  a  number  of  headings  as  shown: 

(1)  General  Service  Knowledge. 

(2)  General  Management. 

(3)  Australian  Government. 

(4)  International  Affairs. 

(5)  Defence  Management. 

( 6 )  RAAP  Management . 

(7)  RAAF  Operations. 

(8)  Military  and  Naval  Operations. 

5.  The  survey  was  administered  to  a  40  percent  sample  of 

RAAF  officers  in  the  ranks  of  PLTOFF  to  GPCAPT  during  May  1979. 
In  all,  1230  respondents  completed  the  survey  and  the  data 
obtained  from  these  respondents  were  used  for  analysis. 


SURVEY  ANALYSIS 


6.  The  task  inventory  data  were  analysed  firstly  by  a  CODAP 

programme  called  DIAGRAM  which,  as  the  name  implies,  produces  a 
computer  printout  in  the  form  of  a  diagram.  The  diagram  is 
shown  at  Annex  A.  The  programme  grouped  officers  into  functional 
groups  based  on  the  similarity  of  tasks  they  perform.  The  back¬ 
ground  information  mentioned  earlier  was  used  to  identify  the 
various  groups.  From  the  diagram,  the  following  conclusions 
can  be  drawn: 

a.  Senior  officers  perform  a  different  set  of 
tasks  from  junior  oificers  although  there 
are  some  tasks  that  are  contained  in  each  set. 
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b.  Officers  have  grouped  according  to: 

( 1 )  Rank. 

(2)  Category. 

(3)  Number  of  subordinates. 

(4)  Current  employment.  (eg,  staff  officers 
have  grouped  separately  from  unit 
operatives) . 

c.  One  third  of  RAAP  officers  do  not  have  any 
subordinates.  The  categories  in  which  junior 
officers  are  unlikely  to  have  subordinates  are 
General  Duties  Branch  categories  of  pilot, 
navigator,  air  electronics  officer,  Air  Traffic 
Control  and  Air  Defence. 

7.  Prom  a  training  standpoint,  the  diagram  indicates  that 
there  are  different  training  requirements  for: 

a.  junior  officers  without  subordinates  (unit 
operatives  or  instructors); 

b.  junior  officers  with  subordinates  (unit 
flight  or  section  commanders); 

c.  flight  lieutenants  and  squadron  leaders  in 
staff  appointments ;  and 

d.  senior  officers  who  are  commanding  officers, 
squadron  commanders  in  large  units  or  staff 
officers. 

Computer  printouts  of  the  tasks  performed  by  the  various  officer 
groups  have  been  obtained. 

8.  The  question  now  arises:  How  does  the  OET  design  team 
use  this  information  to  determine  officer  education  and  training 
needs?  Ideally,  the  RAAP  Directorate  of  Personnel  Officers  (DPO) 
should  be  able  to  plan  all  officers'  careers.  If  this  was  the 
case,  then  officers  could  be  trained  appropriately  just  prior  to 
their  being  posted  to  a  particular  appointment.  However,  while 
ideal,  the  level  of  career  planning  required  is  difficult  to 
achieve  in  practice.  Therefore,  instead  of  providing  training 
for  each  officer  as  required,  the  team  had  to  identify  times  in 
terms  of  ranks  and  seniority  within  ranks  at  which  training  was 
appropriate  for  a  significant  percentage  of  officers.  We  have 
determined  a  set  of  courses  and  training  programmes  which,  we 
believe,  will  provide  the  RAAP  with  an  effective  and  progressive 
officer  education  and  training  programme.  Since  the  team's 
report  has  not  yet  been  finalised,  the  proposed  new  scheme  cannot 
be  discussed  in  detail;  suffice  to  say  that  it  is  similar  in 
concept  to  the  1973/74  Working  Party  scheme  but  that  it  differs 

in  types  of  training,  timing  of  training  and  course  content. 
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9.  Having  determined  the  most  suitable  training  programme, 
we  then  determined  what  tasks  should  be  trained  at  each  level. 

This  has  been  done  by  using  a  Factor  Summary  (FACSUM)  printout, 
a  sample  page  of  which  is  shown  at  Annex  B.  As  an  example, 

Task  1  -  Write  Official  Letters  -  is  performed  by  a  high  percentage 
of  all  officer  ranks.  Therefore,  training  should  be  provided  at 
the  officer  initial  training  stage.  Also,  since  the  percentage 
of  officers  performing  the  task  increases  at  squadron  leader  rank, 
consolidation  of  that  task  should  be  provided  at  the  senior  flight 
lieutenant  level.  A  similar  process  was  carried  out  for  the 
educational  topics  inventory. 

10.  The  FACSUM  printout  was  not  the  only  information  used  to 
determine  when  training  should  be  conducted.  There  are  many 
other  CODAP  programmes  which  we  have  used  to  clarify  what  tasks 
should  be  taught  and  when.  There  is  insufficient  time  to  discuss 
them  all.  I  can,  however,  mention  one  particular  programme  of 
considerable  use  in  determining  the  training  of  tasks.  Senior 
officers  were  asked  to  rate  each  task  in  the  task  inventory  in 
terms  of  the  difficulty  of  learning  the  task  on  a  1  to  9  relative 
scale.  From  the  programme  printout  we  obtained  the  relative 
difficulty  of  each  task  as  rated  by  over  400  senior  officers. 

The  ratings  were  then  used  to  help  determine  whether  the  training 
given  for  a  task  was  formal,  correspondence  or  on-the-job. 


CONCLUSION 


1 1 „  The  officer  education  and  training  survey  has  provided 

the  design  team  with  the  objective  data  on  which  an  effective 
programme  can  be  produced.  The  survey  has  enabled  the  team  to 
determine  what  training  is  required  and  when  that  training  should 
be  given.  However,  the  survey  cannot,  nor  should  it,  provide  all 
the  answers  for  the  design  of  an  officers  education  and  training 
programme.  There  are  a  number  of  subjective  assessments  that 
have  to  be  made  separately:  for  example,  the  philosophy  of 
initial  training  for  junior  officers  -  indoctrination  or  orienta¬ 
tion;  high  or  low  pressure  course  etc.  The  objective  data, 
though,  has  formed  the  basis  for  the  development  of  what  we 
believe  will  provide  better  education  and  training  of  RAAF 
officers  in  the  future. 


ANNEX  A  to 


RAAF  OFFICER  EDUCATION  AND 
TRAINING  SURVEY 


RAAF  OFFICER  EDUCATION  AND  TRAINING  SURVEY  -  TASK  DIAGRAM 


Description  of  Groups 

GRP  419  Senior  staff  officers  (SQNLDRs,  WGCDRs,  GPCAPTs)  at  Air  Force  Office 
or  Commands. 

GRP  410  Senior  officers  (SQNLDRs,  WGCDRs,  GPCAPTs)  who  are  commanding  officers 
or  squadron  commanders  in  large  units. 

GRP  373  Amalgamation  of  GRP  419  and  GRP  410 

GRP  307  Mainly  SQNLDRs  in  unit  flight  commander  appointments 

GRP  281  Amalgamation  of  GRP  373  and  GRP  307 

GRP  376  Mixed  ranks  mainly  equipment  officers  in  command  staff  appointments. 

GRP  365  Mainly  FLTLTs  who  are  unit  section  or  flight  commanders. 

GRP  230  Amalgamation  of  GRP  376  and  GRP  365 

GRP  161  WGCDRs  and  SQNLDRs  of  GD/INTEL  Categories  in  operations  related 
staff  appointments. 

GRP  79  SQNLDRs  and  FLTLTs  who  do  not  have  subordinates  in  staff  appointments 
at  Air  Force  Office  or  Commands. 

GRP  45  FLTLTs,  mainly  nurses  in  unit  operative  posts  without  subordinates. 

GRP  76  Junior  officers  of  GD/ATC  Categories  in  unit  operative  posts 
without  subordinates. 

GRP  55  Junior  officers  of  GD/ATC/ED  Categories  in  instructional  posts 
without  subordinates. 

GRP  47  Amalgamation  of  GRP  76  and  GRP  55 
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RELEVANCE  OF  OCCUPATIONAL  ANALYSIS 
TO  MILITARY  TRAINING 


By  Dr  R.E.  Christal 
CODAP  STUDIES'  RELATED  TO  TRAINING 


1 .  There  are  a  number  of  questions  which  need  to  be  answer¬ 
ed  in  order  to  establish  an  effective  training  course:  What 

should  be  trained?  How  should  they  be  trained?  Where  should 
they  be  trained?  When  should  they  be  trained?  Most  training 
research  during  the  last  few  decades  has  been  directed  toward 
the  "How"  question.  Research  on  training  methods  is  important; 
but  the  quality  cf  instruction  makes  little  difference  if 
course  content  is  not  responsive  to  job  requirements.  Data 
collected  with  occupational  surveys  and  analyzed  with  CODAP  can 
provide  direct  information  for  specifying  course  content,  and 
can  be  useful  in  determining  when  training  should  be  given  in 
order  to  be  timely.  Occupational  data  also  can  be  analyzed 
to  answer  specific  questions  posed  by  training  managers.  To¬ 
day,  I  will  summarize  a  few  studies  we  have  conducted  to  address 
management  questions,  and  then, I  will  describe  a  more  systematic 
research  stream  designed  to  provide  a  technology  for  determining 
the  content  of  entry  -level  training  courses. 


MISCELLANEOUS  CODAP  STUDIES  RELATINO  TO  TRAINING 


2.  In  most  Air  Force  occupations,  all  personnel  attend  a 
residence  training  course  before  being  assigned  to  their  first 
job.  However,  there  are  a  few  occupations  in  which  only  a 
subset  of  individuals  are  taught  in  residence  courses,  while 
the  rest  are  given  formalized  training  at  their  first  job 
location.  A  question  was  posed  concerning  whether  training 
in  the  two  settings  was  in  fact  equivalent  in  terms  of  the 
skills  acquired  by  the  recipients.  It  was  not  possible  for  us 
to  measure  skills  directly,  but  we  could  compare  the  two  groups 
in  terms  of  performance  evaluations  and  the  difficulty  of  work 
inherited.  We  hypothesized  that  those  taught  in  the  job 
setting  would  be  assigned  the  more  challenging  tasks  until 
those  arriving  from  the  resident  schools  were  better  known. 
However,  we  felt  that  if  the  resident  schools  provided  better 
training,  the  superior  skills  of  school  graduates  would  event¬ 
ually  be  recognized  and  reflected  in  their  job  assignments  and 
performance  ratings.  Analyzing  data  with  the  CODAP  system 
failed  to  show  significant  differences  between  the  two  groups 
at  any  point  in  time.  A  simple  analysis,  such  as  this,  laid 
to  rest  uneasy  feelings  that  those  trained  in  the  operational 
setting  were  being  short-changed. 

3.  The  Inspector  General  in  charge  of  the  Air  Force  Safety 
Program  wanted  to  know  if  occupational  survey  data  could  be 
analyzed  to  help  him  determine  where  additional  training  on 
safety  practices  should  be  introduced.  He  had  data  files 
containing  information  on  all  significant  accidents  which  had 
occurred  during  the  previous  two-year  period.  Using  these 
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data,  we  were  able  to  determine  the  number  of  accidents 
occurring  in  connection  with  each  task  in  an  occupation. 

Then  we  set  about  to  develop  an  equation  to  predict  accidents. 
Eventually,  we  were  successful  in  predicting  past  accidents 
with  an  equation  which  included  hazard  ratings  collected  from 
supervisors,  the  number  of  individuals  performing  each  task, 
and  the  time  spent  performing  each  task.  Applying  this 
equation,  we  were  able  to  predict  the  accidents  which  had 
occurred .  However,  the  predicted  scores  also  identified 
many  tasks  with  hazard  potential  on  which  accidents  had  not 
occurred.  We  will  hold  these  data  for  two  years  and  see  how 
well  the  equation  predicts  future  accidents.  If  it  proves 
valid,  it  will  be  used  to  guide  safety  training  programs. 

4.  Most  of  you  are  aware  that  the  CODAP  system  is 
normally  used  to  analyze  jobs  which  are  defined  in  terms  of  the 
time  workers  spend  on  tasks  in  their  jobs.  However,  CODAP 

is  a  very  flexible  system,  and  it  can  be  applied  to  other  types 
of  questionnaire  data.  For  example,  the  CODAP  system  was 
used  to  analyze  the  electronic  principles  claimed  to  be  used 
by  workers  in  Air  Force  electronic  maintenance  occupations; 
it  was  used  to  analyze  the  need  expressed  by  officers  for 
instruction  on  professional  education  topics;  it  is  presently 
being  used  to  analyze  the  utilization  of  knowledges  gained  by 
scientists  and  engineers  in  previous  academic  courses;  and  it 
was  used  to  analyze  ratings  by  pilots  on  the  utility  of  sim¬ 
ulators  for  training  various  flight  activities.  In  each  of 
the  above  studies,  the  data  were  used  for  determining, the 
content  of  courses  or  specifying  job  requirements. 

5.  During  the  Vietnam-Thailand  conflict,  there  was  a 
question  as  to  whether  the  Air  Force  special  operations 
course  should  be  expanded.  This  course  covered  such 
activities  on  counter-insurgency,  unconventional  warfare,  and 
civic  actions.  Before  the  decision  was  made,  HQ  USAF  asked 
that  a  special  survey  be  conducted  to  determine  the  extent 

of  Air  Force  participation  in  such  activities.  We  built  a 
very  comprehensive  task  inventory,  but  our  main  difficulty 
was  in  locating  individuals  performing  special  operation  tasks, 
since  they  were  not  specifically  identified  by  occupational 
code.  Our  findings  were  that  most  of  these  activities  were 
being  performed  by  Army  personnel,  and  that  the  Air  Force  had 
a  very  low  level  of  participation.  As  a  matter  of  fact,  very 
few  of  the  special  operations  course  graduates  were  utilizing 
their  skills  in  this  area.  The  data  provided  a  clear  basis 
for  decision. 

6.  You  may  be  interested  in  knowing  that  we  conducted 
occupational  surveys  in  Vietnam  and  Thailand  throughout  the 
period  of  conflict.  The  resulting  data  were  used  for  many 
purposes,  but  one  study,  in  particular,  may  be  of  interest  tc 
you.  We  were  asked  to  evaluate  whether  our  training  courses, 
which  were  designed  for  a  peacetime  environment,  had  prepared 
our  personnel  for  the  types  of  tasks  they  encountered  in  the 
combat  zone.  In  addition  to  the  routine  administration  of 
job  inventories,  we  interviewed  hundreds  of  returning  NCOs 
concerning  the  work  tasks  which  their  subordinates  were 
unprepared  to  accomplish.  In  almost  every  occupation,  there 
were  some  tasks  for  which  training  had  not  been  provided. 
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Some  of  the  tasks  were  important;  other  tasks  were  not. 
Loadmasters  had  been  trained  to  load  and  unload  almost  every¬ 
thing  except  ice  and  cattle,  which  were  often  moved  in  Vietnam. 
The  weather  people  knew  how  to  make  weather  forecasts,  but 
they  did  not  know  how  to  set  up  weather  stations.  A  number 
of  operations  had  become  dependent  on  centralized  computers 
which  were  not  available  in  the  combat  zone.  Only  the  old 
hands  had  the  knowledge  to  revert  to  manual  procedures.  In 
the  administrative  area,  the  people  did  not  know  how  to  write 
money  orders  or  separate  mail  for  shipment,  because  these 
activities  had  been  civilianized .  But  despite  some  problems 
in  almost  every  occupation,  the  general  conclusion  was  that 
what  Air  Force  people  did  in  Vietnam  was  not  very  different 
from  what  they  did  in  peacetime  in  the  United  States.  Some 
individuals  had  to  be  quickly  cross-trained  to  shortage  areas, 
such  as  munition  loaders,  but  the  tasks  within  occupations 
were  essentially  those  for  which  individuals  had  received 
training. 

7.  We  presently  are  helping  the  Air  Force  develop  and  test 
a  new  on-the-job  training  program.  At  the  present  time, 
supervisors  vary  in  the  amount  and  type  of  training  they  give 
their  subordinates.  The  goal  is  to  provide  guidance  and 
procedures  that  will  assure  workers  will  receive  thorough 
training  on  critical  tasks.  We  are  presently  analyzing 
occupational  data  with  the  CODAP  system  in  a  sample  of 
specialties  to  identify  the  tasks  in  each  job  type,  and  those 
cutting  across  job  types,  which  should  be  classified  as 
critical. 

8.  The  USAF  operates  a  very  large  program  for  retaining 
individuals  into  new  occupations.  Approximately  10,000  to 
20,000  such  actions  occur  each  year,  due  to  a  variety  of 
causes.  Some  are  retrained  because  they  fail  an  entry-level 
course;  some  because  they  are  promoted  into  an  overage  status; 
some  because  they  are  needed  in  a  new  weapon  system;  some 
because  they  want  to  reenlist  in  the  Air  Force,  and  there  is 
no  room  for  additional  second-term  personnel  in  their  occupa¬ 
tion.  We  have  been  asked  to  find  out  how  well  individuals 

do  in  their  second  occupation.  In  one  study,  we  located 
occupational  data  on  18,000  airmen  who  had  been  cross-trained. 
We  were  able  to  demonstrate  that  there  was  very  little 
difference  between  the  retrainees  and  those  in  their  original 
occupation  on  variables  such  as  utilization  of  talent,  job 
interest,  and  career  intentions.  However,  we  were  able  to 
show  that  in  some  occupations,  when  months  of  military  service 
was  held  constant,  the  retainees  were  behind  in  terms  of  grade 
achievement,  number  of  subordinates  supervised,  and  difficulty 
of  work  inherited.  These  differences  were  significant,  but 
not  large. 

9«  In  a  second  study  on  retraining,  we  have  data  on  40,000 

cases  who  were  retrained  between  1973  and  1977.  Some  of  the 
findings  are  a  little  surprising.  In  academic  achievement, 
cross-trainees  did  better  in  the  entry-level  course  than  those 
who  were  being  trained  for  the  first  time  (holding  aptitude 
constant).  The  more  service  an  individual  had  when  he  went 
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into  the  course,  the  better  he  performed;  therefore,  the 
people  who  retrained  in  mid-career  actually  did  better  than 
those  who  were  retrained  in  the  first  four  years.  Individuals 
who  had  been  retrained  from  a  mechanical  area  to  another 
mechanical  area  performed  better  than  people  who  had  come  from 
the  administrative  or  general  areas.  In  general,  people  who 
trained  in  the  same  aptitude  domain  performed  better  than  those 
who  came  from  other  domains.  The  one  exception  was  that 
individuals  from  electronics  AFSs  did  better  in  retraining 
courses  than  all  other  groups,  including  new  accessions.  This 
is  probably  because  only  individuals  with  excellent  learning 
abilities  tend  to  get  thru  electronics  course  training.  A 
significant  question  to  be  answered  in  this  study  concerns 
the  number  of  points  on  aptitude  scores  that  can  be  waived  for 
cross-trainees.  We  found  that  a  10-point  waiver  was  really 
quite  conservative  and,  in  many  cases,  you  could  waive  as  many 
as  20  points  for  retrainees  to  have  equal  probabilities  of 
succeeding. 


TRAINING  COURSES  FOR  ENTRY  LEVEL  PERSONNEL 


10.  Thusfar,  I  have  briefly  described  a  number  of  short¬ 
term  studies  related  to  training  which  made  use  of  occupational 
information.  Now,  I  would  like  to  describe  how  this 
information  has  and  is  being  used  in  the  Air  Force  to  determine 
the  content  of  entry-level  training  courses. 

11.  For  a  number  of  years,  we  simply  delivered  CODAP 
analyses  outputs  to  the  training  schools  for  them  to  use  as 
they  saw  fit.  These  reports  contained  task  level  job 
descriptions  indicating  the  percent  of  first-termers  performing 
each  task,  which  we  felt  could  be  used  to  eliminate  deadwood 
from  courses  and  to  determine  training  emphasis.  They  also 
contained  information  on  task  difficulty  levels,  job-type 
analyses,  summary  data  on  tools  and  test  equipment  used,  group 
summaries,  and  many  other  outputs.  However,  these  computer 
printouts  looked  so  complicated  that  there  was  a  strong 
tendency  for  the  trainer  to  become  confused  and  not  use  the 
information.  Also,  if  no  one  was  complaining  about  a  course, 
there  was  no  urge  to  change  it.  I  think  it  is  essential  that 
trainers  become  ego  involved  with  the  whole  process  of  develop¬ 
ing  the  inventory  and  collecting  the  data.  You  cannot  just 
deliver  the  computer  output  and  expect  them  to  use  it.  On 
occasions,  we  found  almost  unbelieveable  discrepancies  between 
training  courses  and  what  was  happening  on  the  job.  For 
example,  in  one  career  development  course,  25  percent  of  the 
course  content  was  on  tasks  that  were  not  being  performed. 

Yet,  the  course  was  not  changed,  even  though  the  school  had 
access  to  the  CODAP  report.  We  finally  called  the  problem 

to  the  attention  of  management,  and  the  course  was  modified. 

12.  After  several  experiences  like  this,  HQ  Air  Training 
Command  (ATC)  established  guidance  for  the  use  of  occupational 
survey  data  in  the  design  and  modification  of  courses.  If 
over  50  percent  of  the  workers  in  an  occupation  perform  a  task 
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in  their  first  assignment,  then  all  would  be  trained  to  full 
proficiency  on  the  task;  if  between  30  percent  and  50  percent 
perform  it,  then  all  would  be  trained  to  a  point  where  they 
could  perform  the  task  under  close  supervision;  if  fewer  than 
30  percent  perform  it,  then  training  would  be  limited  to 
providing  general  information  about  the  task.  The  guidance 
allowed  for  exceptions  when  justified,  and  it  was  a  good 
thing  it  did.  There  are  a  lot  of  reasons  why  many  tasks 
should  be  taught  even  though  fewer  than  30  percent  of  the 
people  perform  them.  On  the  other  hand,  there  are  tasks  which 
90  percent  of  the  people  perform  which  do  not  need  to  be  includ¬ 
ed  in  a  training  course,  because  they  are  so  simple. 

Eventually,  ATC  asked  us  to  conduct  research  on  a  systematic 
method  of  establishing  task  training  priorities  for  entry- 
level  training  courses.  First,  we  looked  for  factors  that 
needed  to  be  considered.  The  trainers  gave  us  some  useful 
suggestions.  Eventually,  we  produced  a  set  of  factors 
hypothesized  to  be  relevant  as  follows: 

a.  probability  of  encountering  a  task; 

b.  consequences  of  inadequate  performance; 

c.  probability  of  inadequate  performance; 

d.  perishability  of  skills; 

e.  task  difficulty; 

f.  need  for  emergency  performance; 

g.  trainability  of  skill;  and 

h.  cost-effectiveness  of  residence  school  training. 

We  found  that  some  tasks  with  low  probability  of  performance 
had  to  be  taught  because  of  the  high  consequences  of  inadequate 
performance.  Examples  are  mouth-to-mouth  resuscitation  and 
external  heart  massage.  Although  the  probability  of  performing 
such  tasks  during  a  first  enlistment  may  be  only  one  in  a 
hundred,  the  consequences  of  inadequate  performance  is  severe. 
Consequences  are  measurable  in  terms  of  loss  or  damage  to 
equipment  and  the  loss  of  life.  When  probability  of  inadequate 
performance  is  high,  a  task  should  be  included  in  a  curriculum. 
When  the  perishability  rate  of  a  skill  is  high,  training  should 
not  take  place  until  the  skill  is  actually  required.  Task 
difficulty  is  concerned  with  the  amount  of  time  required  to 
learn  to  perform  adequately.  Originally,  we  had  the  concept 
of  emergency  performance  because  in  some  situations  you  do  not 
have  time  to  find  out  how  to  perform  a  task.  With  regard  to 
trainability,  we  knew  that  there  were  some  skills,  such  as 
verbal  communication  skills,  that  were  not  a  matter  of  training 
but  a  matter  of  selection.  Finally,  we  tried  to  determine  the 
cost-effectiveness  of  training  tasks  in  residence  courses. 

Well,  we  collected  data  on  two  occupations  using  those  factors 
and  did  some  regression  analysis.  We  found  that  we  could 
obtain  reliable  measures  on  all  the  factors,  but  there  were 
some  interesting  redundancies.  Task  difficulty  and  skill 
perishability  were  almost  synonymous.  The  more  difficult  a 
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task,  the  faster  the  skill  perishes.  So  we  felt  that  we  did 
not  need  skill  perishability  as  well  as  task  difficulty.  The 
paradox  is  that  the  very  things  that  are  the  most  difficult 
are  going  to  perish  the  fastest,  and  this  makes  the  training 
decisions  even  tougher.  Also,  in  many  instances,  task 
difficulty  is  almost  synonymous  with  the  probability  of  inadequ¬ 
ate  performance.  The  need  for  emergency  performance  proved 
to  be  a  bad  factor.  People  were  confused  as  to  whether  the 
task  itself  was  an  emergency  or  whether  it  took  place  in  an 
emergency  setting.  For  example,  we  got  high  ratings  on  tasks 
like  directing  traffic  at  the  scene  of  a  fire.  However,  the 
most  consequential  problem  was  associated  with  the  cost- 
effectiveness  of  training  a  residence  course.  People  could 
not  agree  as  to  whether  training  should  be  on-the-job  or 
residential,  even  though  there  was  no  disagreement  about  the 
need  to  train  for  particular  tasks.  The  lesson  we  learned 
was  that  first  you  determine  the  priority  of  teaching  a  task 
for  entry-level  people.  When  that  question  is  settled,  you 
decide  whether  it  should  be  taught  on-the-job  or  in  resident 
school. 

13.  Eventually,  we  reached  the  point  where  we  had  the 
following  four  factors: 

a.  probability  of  encountering  a  task; 

b.  task  delay  tolerance; 

c. 

c.  task  difficulty;  and 

d»  consequences  of  inadequate  performance. 

The  first  factor,  probability  of  encountering  a  task,  is 
reflected  in  the  members  performing  column  on  the  job  descrip¬ 
tions.  The  second  one  is  task  delay  tolerance;  there  are 
certain  tasks  which,  when  encountered,  must  be  performed  at 
once.  Otherwise,  the  opportunity  rapidly  passes.  A  lot  of 
medical  tasks  are  of  that  type.  This  is  an  important  factor 
because,  with  many  tasks,  people  must  be  ready  to  perform  them 
at  any  point  in  time.  The  third  and  fourth  factors  were 
retained  from  the  original  list.  Our  goal  was  to  develop  a 
single  equation  which  would  combine  these  factors  into  a 
composite  indicating  task  training  priorities  for  entry-level 
courses.  Using  instructors'  and  supervisors'  ratings  as 
criteria,  we  developed  equations  in  30  occupations.  Our  hope 
was  the  standard  score  weights  for  the  predictor  variables 
would  be  highly  similar  in  all  occupations  so  that  we  could 
define  a  single  equation.  Unfortunately,  the  weights  across 
occupations  were  not  constant.  We  also  began  to  recognize 
that,  even  if  we  had  a  good  composite  equation,  it  would  be 
difficult  to  apply.  It  would  require  obtaining  supervisory 
ratings  on  four  factors  for  400-800  tasks  in  every  occupation. 
Based  upon  past  research,  we  knew  that  any  supervisor  should 
not  be  asked  to  rate  tasks  on  more  than  one  factor.  Thus, 
we  needed  200  raters  in  each  occupation  to  implement  the  system. 
For  a  number  of  occupations,  this  was  more  than  existed. 

Finally,  we  gained  insight  into  a  simpler  and  yet  highly  defens¬ 
ible  approach.  In  about  80  percent  of  the  occupations,  we  had 
data  indicating  there  was  a  very  high  inter-rater  agreement 
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among  supervisors  concerning  task  training  priorities. 
Furthermore,  we  could  demonstrate  that  these  recommendations 
were  based  upon  consideration  of  factors  with  high  face  valid¬ 
ity  (probability  of  performance,  consequences  of  inadequate 
performance,  task  delay  tolerance,  and  task  difficulty). 

Where  there  was  high  inter-rater  agreement  among  supervisors 
concerning  training  priorities,  these  factors  correlated  in  the 
90s  with  training  recommendations.  Thus,  it  looked  like  we 
had  an  acceptable  system.  We  could  simply  collect  direct 
recommendations  of  supervisors  concerning  training,  and  we 
could  defend  those  recommendations  by  our  research.  But 
remember  that  I  am  referring  to  about  80  percent  of  the  Air 
Force  specialties.  With  the  remaining  20  percent,  there  is 
a  special  problem.  There  are  the  specialties  where  we  do  not 
obtain  agreement  among  supervisors  as  to  what  should  be  trained 
We  have  hypothesized  that  the  primary  reason  for  the  disagree¬ 
ment  stems  from  the  gross  differences  in  the  work  performed 
in  the  various  Air  Force  commands.  The  supervisors  are 
rightly  pushing  those  tasks  which  they  have  been  experiencing 
during  their  careers  in  their  own  commands.  If  that  is  the 
case,  we  may  end  up  with  an  output  that  shows  the  recommended 
training  emphasis  for  every  task  by  command  groupings.  With 
those  tasks  on  which  everybody  is  in  agreement  across  commands, 
we  will  develop  a  core  curriculum.  For  tasks  unique  to  cer¬ 
tain  commands,  special  courses  or  OJT  may  be  needed.  But,  at 
this  stage,  that  is  still  a  hypothesis. 

14.  Another  matter  of  concern  is  whether  we  should  train 
on  what  people  are  doing  or  on  what  they  should  be  doing. 

After  looking  at  a  lot  of  data  from  many  occupations,  my  person¬ 
al  opinion  is  that  people  in  the  Air  Force  are  doing  what  has 
to  be  done  (and  if  you  eliminate  the  combat  arms  in  the  Army, 

I  think  the  same  thing  would  be  found  true).  People  repair 
aircraft  because  they  break  down,  and  cooks  prepare  meals 
because  people  have  to  eat.  However,  I  would  agree  that 
occasionally  the  occupational  data  tell  us  that  people  are  not 
doing  what  they  should  be.  You  do  not  stop  training  on  a 
task  just  because  people  are  not  performing  it,  especially  if 
they  should  be.  For  example,  in  the  preventative  medicine 
area,  one  person  at  each  Air  Force  Base  was  supposed  to  make 
a  series  of  tests  to  protect  the  health  of  the  community. 

A  survey  showed  tnat  no  one  was  performing  a  particular  test. 

An  investigation  revealed  that  the  task  was  not  being  done 
because  people  did  not  know  how  to  do  it.  Now,  if  you  were 
blindly  using  the  data,  you  might  say  that  no  training  was 
required,  which  is  as  far  from  truth  as  you  could  get. 

Another  good  example  is  in  the  equipment  operator  area.  We 
discovered  that  people  were  well  trained  to  operate  all  types 
of  equipment,  but  what  they  actually  did  was  mow  the  grass, 
water  the  flowers  and  shovel  the  snow.  This  occurred 
because  the  Air  Force  was  contracting  jobs  out  or  letting  the 
civilian  employees  do  the  more  difficult  jobs,  and  the  airmen 
were  doing  the  leftovers.  In  this  case,  there  was  nothing 

wrong  with  the  course.  If  the  Air  Force  wants  an  equipment 
operator  capability,  then  it  will  have  to  change  the  job,  not 
the  course.  In  another  example,  we  found  that  a  large  number 
of  supervisors  in  Medical  Administration  were  recommending  that 
more  training  be  given  in  filling  out  certain  types  of  forms. 


Now,  if  you  looked  at  the  task  inventory  output,  you  find  that 
very  few  people  at  the  junior  level  are  filling  out  those  forms. 
I  think  the  supervisors  were  trying  to  say  that  they  had  more 
important  things  to  do  than  fill  out  forms,  and  they  wanted 
to  delegate  those  tasks  down  to  the  junior  airmen. 

15.  Large  scale  cuts  in  training  budgets  have  often  led  to 
difficult  decisions  concerning  cuts  in  training  courses.  In 
the  past,  about  the  only  thing  that  could  be  done  was  to  ask 
each  course  to  take  its  fair  share  of  the  cut.  Recently,  the 
Air  Force  Training  Command  received  a  multi-million  dollar 
cut  in  budget,  and  this  time  it  was  decided  that  occupational 
survey  reports  would  be  used  for  decisions  concerning  reductions 
in  training.  Even  old  survey  reports  were  dusted  off  and  used 
for  the  occasion.  While  the  occupational  survey  data  were 
used  to  make  decisions  concerning  the  course  reductions,  they 
were  not  responsible  for  the  cost  savings.  The  cost  savings 
were  due  to  the  budget  cut.  However,  we  learned  a  lot  from 
this  exercise  concerning  how  survey  data  should  be  organized 
to  make  them  more  useful  to  trainers.  We  now  provide  a  single 
CODAP  report  which  is  specifically  designed  for  course  review. 
Each  task  statement  appears  only  once,  with  training-relevant 
information  printed  beside  it.  The  tasks  are  organized  into 
blocks  using  a  "mapping  process"  by  which  tasks  in  an  inventory 
are  "mapped"  into  the  present  course  outline.  The  tasks 
associated  with  each  block  of  instruction  appear  together, 
ordered  in  terms  of  mean  training  emphasis  ratings  from  field 
supervisors.  Also  presented  are  data  on  other  factors,  such 
as  task  difficulty  and  percent  of  first-termers  performing. 

Tasks  which  are  not  covered  by  the  present  course  of  instruction 
are  listed  at  the  bottom  of  the  report,  ordered  in  terms  of 
mean  training  emphasis  recommendations.  Each  course  is  re¬ 
viewed  by  a  team  made  up  of  course  personnel,  representatives 
of  the  using  commands,  and  the  occupational  analyst  who 
conducted  the  survey.  The  analyst  spends  most  of  the  first 
session  describing  to  the  review  team  how  the  survey  was 
conducted  and  how  the  values  on  the  CODAP  report  were  derived. 
The  team  then  reviews  the  entire  course  of  instruction  using 
the  CODAP  report  and  makes  recommendations  concerning  course 
modifications.  Sometimes,  these  recommendations  lead  to 
lengthening  the  course  in  order  to  accommodate  important  train¬ 
ing  not  covered.  At  other  times,  it  leads  to  course  reductions 
through  the  elimination  of  training  not  being  utilized.  In 
all  instances,  reviews  have  led  to  substantial  course  modifica¬ 
tions  through  a  shifting  emphasis  in  accordance  with  field 
recommendations  reported  in  the  CODAP  output.  We  believe 
that  this  team  approach,  using  the  revised  CODAP  output,  has 
improved  the  utilization  of  occupational  data  by  trainers  by 
an  order  of  magnitude. 
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Squadron  Leader  D.  Thomson,  BSc,  DipEd 
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1 o  The  aim  of  this  paper  is  to  show  and  describe  some  of  the 

main  computer  outputs  that  analysts  and  management  personnel  see 
and  use.  CODAP,  as  the  Australian  Defence  Force  knows  it, 
currently  contains  over  sixty  general  and  special  purpose  programmes. 

2.  Table  1  shows  the  general  areas  of  programme  operation. 

What  is  not  shown  are  the  names  of  the  programmes  that  fall  into 
each  area. 


THE  CODAP  SYSTEM 


3.  The  Data  Prepatory  Programmes  take  the  raw  optical  scanned 

or  key  punched  data  and: 

a.  re-organize  it  into  a  form  acceptable  to  all 
later  programmes; 

b.  permit  corrective  editing  directly  by  CODAP 
cell  staff; 
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c.  check  for  incumbent  identity  number  and  number  of 
sheets  of  data,  etc;  and 

d.  prepare  lists  of  duties,  tasks  and  background 
variables  ready  for  matching  with  computed 
data  output  in  later  programmes. 

4.  These  programmes  are  of  such  a  specialist  and  technical 
nature,  that  they  will  not  be  discussed  further.  But  suffice  to 
say,  that  the  accuracy,  validity  and  reliability  of  the  data  from 
later  CODAP  programmes  is  only  as  good  as  the  input  data  from  the 
raters.  Poor  inventory  preparation  -  confusing,  poorly  designed, 
ambiguous  questions  and  task  statements  -  result  in  the  rejection 
of  large  numbers  of  raters  and/or  subject  data  outputs.  Some  of 
the  more  important  Time  Spent  programmes  will  now  be  discussed. 

5.  The  Grouping  Programmes  take  each  incumbent's  raw  data 
responses,  convert  each  task  rating  into  a  percent  time  spent  for 
that  incumbent  and  compute  an  overlap  matrix.  Each  incumbent 
shows,  through  his  ratings,  that  his  job  is  different  from  other 
jobs,  either  by  the  tasks  he  does  or  does  not  rate,  or  by  his 
emphasis  as  to  the  amount  of  time  he  spends  on  each  task  he 
performs. 

6.  Following  the  computation  of  the  overlap  matrix  a 
clustering  programme  is  applied  which  combines  cases,  according  to 
their  similarity  index,  by  stages  until  all  cases  have  been  merged 
into  a  single  group. 

7.  The  Diagram  Programme  uses  the  data  from  the  other 
clustering  programmes  to  generate  a  tree-like  diagram  that 
graphically  displays  which  groups  were  merged  during  the  grouping 
process.  Annex  A  shows  the  diagram  for  a  sample  of  Army 
Engineering  Aircraft  Trades.  Each  group  of  jobs  is  represented 
by  a  rectangular  block  of  data  containing  the  most  important 
information  about  that  group.  Table  2  describes  a  small  portion 
of  that  printout. 

SECTION  OF  THE  ARMY  ENGINEERING  AIRCRAFT  TRADES  DIAGRAM 
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8.  Using  the  last  block  of  data  in  Annex  A  as  an  example, 
the  numbers  tell  us: 

0024  =  Group  Number.  This  group  was  formed  by  the 

merging  of  the  two  smaller  groups  0027  and  0032. 

A  description  of  the  job  performed  by  the  average 
member  of  this  group  can  be  obtained  from  another 
CODAP  programme  which  will  be  discussed  later. 

0040  =  The  number  of  members  in  this  group,  ie  this 

group  was  made  from  clustering  the  two  groups  of 
19  and  21  members. 

0071- 

0110=  The  range  of  K-Path  numbers  exclusively  allotted 
to  the  members  of  this  group.  These  numbers 
uniquely  identify  each  member  so  that  other 
CODAP  programmes  can  refer  to  these  members. 

44.4  =  The  average  percentage  of  overlap  between  all 

possible  pairings  of  the  members  of  group  27  with 
the  members  of  group  32.  (399  averages  = 

N1  x  N2.) 

49«2  =  The  average  percentage  of  overlap  within  all 

possible  pairs  of  individuals  within  the  final 
combined  group  24.  (780  averages  = 

N1  +  N2  -  1 

>  (N1  +  N2  -  1).) 

1 

9.  The  Group  Job  Description  Programmes  have  been  written 

to  enable  the  computer  to  generate  composite  Job  Descriptions  made 
up  of  duties  and  tasks  performed  for  any  group  of  individuals.  By 
nominating  a  group  from  the  diagram  -  and  thus  a  range  of  K-Path 
numbers  -  a  Group  Job  Description  at  task  level  or  at  duty  level, 
or  both,  can  be  produced  for  the  average  job  performed  by  the  member 
of  that  group. 

10.  As  an  example.  Annex  B,  the  Job  Description  for  RAAF 
Armament  Fitters  will  contain: 

a.  the  percentage  of  members  of  that  group  that 
perform  each  task; 

b.  the  average  percentage  of  time  spent  by  those  members 
in  the  group  who  actually  perform  each  task; 

c.  the  average  percentage  of  time  spent  on  each  task 
by  all  members  of  the  group;  and, 

d.  the  cumulative  sum  of  the  average  percentage  of 
time  spent  by  all  members. 


Other  background  variables  may  be  used  to  characterize  and  thus 
nominate  membership  of  a  group. 
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10.  The  Group  Difference  Programme  shows  how  one  Job  Group 
Description  differs  from  another  Job  Group  Description.  It  can 
suppress  the  printing  of  tasks  where  the  differences  are  below 
some  nominated  minimum  value,  and  also  print  the  tasks  in  order  of 
any  of  the  columns  of  data.  The  distinct  difference  in  tasks 
performed  by  Job  Groups  27  and  32  in  the  Army  Tradesmen  Study  is 
shown  at  Annex  C.  At  the  top  of  Annex  C  are  the  tasks  that  have 
large  amounts  of  time  spent  on  them  and  are  performed  by  many 
members  of  Group  27.  These  tasks  are  typically  those  performed 
by  Engines  Tradesmen.  But  working  from  the  bottom  upwards,  those 
tasks  are  supervisory  in  nature  -  and  in  fact  we  find  Group  32  is 
made  up  of  corporal  engine  fitter  supervisors. 

11.  Job  Group  Difference  Descriptions  can  also  be  obtained  from 
any  two  groups  of  individuals  that  have  been  identified  in  terms  of 
background  variables.  For  example;  one  can  obtain  a  description 
of  the  differences  in  tasks  being  performed  by  the  group  of  people 
who  find  their  work  interesting  and  those  who  find  their  work  dull; 
or  the  difference  between  Corporal  RAAF  Clerks  and  Corporal  Army 
Clerks  as  long  as  the  same  task  inventory  has  been  used. 

12.  The  Print  Variables  Programme  generates  a  report  of  sel¬ 
ected  variables  for  selected  groups  of  cases.  When  the  report  is 
for  the  whole  sample  of  raters  and  is  in  K-Path  order,  it  is 
especially  useful  in  identifying  similarity  of  background  charact¬ 
eristics  for  groups  formed  by  the  clustering  process.  This 
capability  aids  the  analyst  in  discerning  which  groups  on  the 
diagram  printout  are  most  susceptable  to  definition  as  meaningful 
job  types  and  it  is  also  useful  in  understanding  and  explaining 
the  worker  composition  of  a  selected  job  group. 

13.  A  section  of  the  report,  in  K-Path  order,  is  reproduced 
in  Table  3,  for  groups  27  and  32  (that  make  up  Group  24)  of  the 
Army  Study. 


RAEME  AIRCRAFT  TRADES  -  GENERAL  BACKGROUND  INFORMATION 
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14.  The  first  column  is  the  K-Path  number  for  each  case.  It 

is  computer  allotted  so  that  the  individuals  in  the  group  who  have 
the  most  similar  jobs  have  adjoining  K-Path  numbers.  But  simil- 
arily  of  the  individual’s  jobs  is  reflected  in  the  other  columns 
of  Personnel  Background  Information.  These  columns  show 
similarity  in: 

Column  2.  The  employment  code  number  or  mustering, 
where  1  =  Aircraft  Artificer; 

2  =  Aircraft  Engines  Fitter  and 

3  =  Airframe  Fitter  etc. 


Column  3.  Rank,  where, 

1:1  =  Craf tsman  Substantive  -  Croup  27 

2:1  =  Lance  Corporal  Substantive 

3:1  =  Corporal  Substantive  -  Croup  32 

Columns  4.5.6  =  Time  in  the  Army,  in  that  rank  and  at 

their  present  unit  respectively  -  shows 
the  more  senior  personnel  in  Croup  32. 

Last  Column.  Is  the  number  of  tasks  performed  by  each 
individual  of  the  possible  600  odd  tasks 
in  the  inventory  -  again  the  more  senior 
people  in  Croup  32  stand  out. 

15.  This  report  could  have  been  arranged,  for  example,  in 
ascending  order  of  the  incumbent’s  job  interest,  ranging  from  very 
dull  to  very  interesting.  That  re-ordered  display  may  highlight 
particular  clusters  of  uninteresting  jobs  at  a  particular  unit,  or 
clusters  of  personnel  that  have  not  been  cross  trained  -  or  are 
under  employed  in  terms  of  the  number  or  type  of  tasks  they  perform. 

16.  The  programmes  discussed  so  far  have  been  orieted  towards 
identifying  and  describing  the  types  of  jobs  in  an  occupational 
category  such  as  a  mustering.  But  there  is  another  set  of  program¬ 
mes  which  manipulate  information  describing  work  tasks,  rather  than 
jobs  or  persons.  These  programmes  assist  in  the  analysis  of 
factors  -  call  task  factors  -  which  describe  the  nature  and 
characteristics  of  each  task,  for  example  the  difficulty  for  any 
new  tradesman  to  learn  to  do  the  task  staisfactorily. 

17.  The  Programme  REXALL,  is  designed  to  compute  the  mean  and 
standard  deviation  of  each  task  from  a  group  of  expert  opinions 

as  to  the  nature  of  those  tasks.  For  example,  a  group  of  27 
specialist  Non  Commissioned  Officers,  using  a  1  to  9  point  scale, 
may  rate  each  task  as  to  the  probable  consequences  of  not  performing 
each  task  satisfactorily,  irrespective  of  who  may  normally  do 
those  tasks. 

18.  Annex  D  shows,  for  the  RAAF's  Electrical  Fitter  mustering, 
some  of  the  tasks  from  that  inventory  arranged  in  descending  order 
of  adjusted  means  of  the  consequences  factor.  The  strength  with 
which  this  averaged  expert  opinion  can  be  accepted  may  be  judged 
from  the  inter-rater  reliability  coefficient,  which  can  be  loosely 
defined  as  the  concensus  of  agreement  between  the  raters.  For 
this  example  it  was  .95  which  is  excellent.  The  Programme  points 
out  the  rater  who  is  non-co-operative  or  misreads  the  instructions 
and  uses  the  scale  in  the  reverse  sense.  The  Programme  can  also 
adjust  each  rater's  total  opinion  back  to  a  common  mean,  ie  it 
removes  rater  bias. 

19.  Finally  the  Task  Factor  Summary  Programme  can  gather  all 
the  factors  that  describe  the  nature  of  each  task  -  along  with  the 
earlier  mentioned  task  performance  factors  -  and  tabulate  them  in 
such  an  order,  that  training  course  design  personnel  can  make 
objective  decision  as  to  what  training  should  be  given,  how  much 
training  should  be  given,  and  when. 


20.  Annex  D  shows,  for  the  RAAP  Electrical  Fitter,  a  small 
section  of  the  task  inventory  with  a  number  of  task  performing 
factors  AC  performing  #  LAC  performing)  and  task  factors 
(consequences  of  inadequate  performance,  task  learning  difficulty, 
and  the  need  for  immediate  performance).  Training  Design  Managers 
and  personnel  responsible  for  maintenance  of  job  skills  would  have 
to  weigh  the  different  factors  against  each  other. 

21 .  Thank  you,  copies  of  this  paper  are  available  in  the 
display  room,  along  with  a  display  of  computer  printouts. 


ANNEX  A  to 
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PREPARATION  OF  INVENTORIES 


Warrant  Officer  Class  One,  W.J.  Noble  (MERIT) 


INTRODUCTION 


1 .  The  aim  of  this  paper  is  to  outline  the  stages  and 

procedures  applied  by  the  Job  Analyst  -  Military  Employments 
Research  and  Information  Team  (MERIT)  in  the  preparation  of  task 
inventories.  A  Task  Inventory  being  a  comprehensive  check  list 
of  all  the  activities  an  incumbent  of  a  particular  job  may  be 
expected  to  perform. 


TASK  INVENTORY  STAGES 

2.  There  are  four  (4)  stages  to  be  considered  in  the 

preparation  of  task  inventories  and  they  are  as  follows: 

a.  planning  conference, 

b.  research  of  data, 

c.  task  inventory  construction  conference,  and 

d.  task  inventory  compilation. 

PLANNING  CONFERENCE 


3.  The  survey  planning  conference  is  arranged  to  formulate 

the  objectives  for  the  survey  and  to  establish  the  initial  planning 
guide  lines.  At  this  meeting  the  following  points  are  established 

a.  the  specific  aims  of  the  survey  (survey  objectives), 

b.  information  needs  which  the  survey  may  satisfy, 


c.  sponsoring  directorate  responsibilities,  which 
include : 


(1)  provision  of  personnel  for  inventory 
compilation  and  review  panels, 

(2)  composition,  location  and  proposed  dates 
for  inventory  construction. 


4.  At  the  completion  of  this  meeting  the  Job  Analyst  is  able 

to  establish  a  timetable  for  all  stages  of  the  survey  to  facilitate 
planning  and  actions  to  meet  the  target  date  set  for  the  survey. 
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4  RESEARCH  OF  DATA 

5.  When  the  requirements  for  a  survey  have  been  identified 

the  Job  Analyst  as  part  of  his  planning  has  as  research  sources: 

a.  previous  MERIT  surveys, 

b.  employment  specifications, 

c.  similar  surveys  conducted  by  other  services,  and 

d.  trade  structure  printout. 

TASK  INVENTORY  CONSTRUCTION 


6.  The  task  inventory  is  used  to  identify  the  employment  areas 
and  tasks  applicable  to  Army  trade.  With  the  use  of  rating  scales 
the  task  inventory  can  identify  the  time  spent,  involvement, 
importance  and  immediacy  factors  governing  the  tradesman. 

7.  The  Job  Analyst  with  the  expert  assistance  of  the  panel 
members  constructs  the  task  inventory  in  the  following  stages: 

a.  Establishes  the  areas  of  responsibility  (trade  and 
misc)  of  all  members  at  each  rank  level  within  the 
trade . 

b.  From  these  select  relevant  duty  areas  and  by 
discussion  construct: 

(1)  The  root  tasks  which  are  the  task  or  tasks 
applicable  to  each  duty  area. 

(2)  By  discussion  and  questioning  of  panel  members 
ascertain  the  elements  which  are  the  actions 
performed  to  complete  a  root  task. 

(3)  Establish  the  miscellaneous  trade  tasks  (those 
that  do  not  fit  into  a  particular  duty  area). 
These  are  included  in  a  separate  area  at  the  end 
of  the  inventory. 

(4)  The  requirement  for  background  information  which 
is  a  description  of  the  soldier  in  regard  rank, 
length  of  service,  time  in  employment  and  other 
such  detail. 


is. 
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TASK  INVENTORY  CONSTRUCTION 

I 

ESTABLISH  AREAS  Of  RESPONSIBILITY 

TRADE  AND  MISCELLANEOUS  AREAS 
IN  WHICH  MEMBERS  ARE  EMPLOYED 

I 

SELECT  DUTY  AREAS 

- 1 - 

IDENTIFY  MAJOR  QR  ROOT  TASKS 
TASK  OR  TASKS  WHICH  CONSTITUTE  A  DUTY  AREA 

I 

ESTABLISH  ELEMENTS 

ACTIONS  PERFORMED  TO  COMPLETE  MAJOR  OR  ROOT  TASK 

I - 1 

MISCELLANEOUS  TASKS  BACKSROUND  INFORMATION 


MAJOR  POINTS  FOR  CONSIDERATION  IN  CONSTRUCTION 


8.  During  the  construction  stage  the  Job  Analyst  must  consider 

at  all  times  the  following  points  in  relation  to  the  trade  being 
surveyed : 


a. 

how  are  members  employed  in  the 

trade, 

b. 

day  to  day  job  functions, 

c . 

role  of  various  ranks, 

d. 

level  of  root  task, 

e . 

level  of  elements.  Are  they  skill  or  knowledge 

and 

f . 

task  statement  should  begin  with 
action  word. 

present  tense 

i 
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TASK  INVENTORY 


MILITARY  POLICEMAN 


DETENTION 

AND 

CORRECTION  DUTIES 

INVESTIGATIONS 

TRAFFIC  CONTROL  ESCORT  DUTIES 
DUTIES 

ROOT  TASKS 

CONDUCT  INITIAL  CONDUCT  MINOR 

INVESTIGATIONS  INVESTIGATIONS 

I 

ELEMENTS 

RECEIVE/RECORD  AND  IDENTIFY  ALLEGED  OFFENCE 
SECURE  CRIME  SCENE 

RECORD  DETAILS  OF  VITNESSES/COMPLAINANTS 

ADVISE  DUTY  OFFICER  OF  PROGRESS  ON  INITIAL 

INVESTIGATION 

HAND  OVER  INVESTIGATION  TO  SIS/CIVIL  POLICE 
COMPILE  AND  SUOMIT  OCCURRENCE  REPORTS 


COMPILATION 


9.  On  completion  of  the  task  inventory  construction  the  Joh 

Analyst  is  responsible  for  the  compilation,  and  prior  to  commence¬ 
ment  ascertains  the  survey  method  to  be  used,  and  if  layout  to  be 
used  is: 

a.  punch  card  method,  or 

b.  optical  scanning  sheets  (.OPSCAN) . 


PART  A  -  TASK  INVENTORY 


a.  Using  the  scales  shown  in  the  response 
guide,  rate  only  those  tasks  ticked  in 
the  *P*  column. 

b.  Use  Occupational  Analysis  Response  Sheet 
01  for  consecutive  tasks  1-100. 


TASK 

DUTY  A  -  BAR  DUTIES 

Perform  Bar  Hvoiene 

1.  Wash  glasses  by  hand 

2.  Clean  bar  equipment 

3.  Clean  bar  area  (shelves,  counter,  floor  etc) 

Perform  Bar  Operation/Maintenance 

4.  Set  up  beer  dispensing  equipment 

5 .  Maintain  beer  dispensing  equipment 

0.  Operate  beer  dispensing  equipment 

7.  Perform  opening/closing  procedure  for  bar 

operation 


10.  The  structured  task  inventory  is  then  drafted  ensuring 

that: 

a.  Task  statements  are  clear  and  understood. 

b.  Task  statements  restricted  to  sixty  (60) 
characters  in  length  for  ease  of  processing. 

c.  Task  statements  rateable  in  terms  of  'time 
spent'  and  other  factors  in  current  use. 

d.  Inventory  restricted  to  twenty  six  (26)  duty 
areas  and  1700  tasks. 


CONCLUSION 


11.  On  comp]°tion  of  the  compilation  stage  in  conjunction 

with  the  previous  steps  detailed  the  inventory  is  completed  in 
draft  form  and  would  at  this  stage  be  reviewed  by  another  panel 
or  by  mail  to  ensure  all  trade  areas  have  been  included  prior  to 
final  drafting. 


THE  APPLICATION  OF  CODAP/OCCUPATIONAL  ANALYSIS  IN  THE  RAN 


Lieutenant  Commander  R.  Hawketts,  RAN 


1 .  The  Navy's  Occupational  Analysis  cell  is  small  in  comparison 
with  units  fulfilling  the  same  role  for  other  Services,  both  in 
Australia  and  overseas.  This  is  not  only  because  the  Navy  itself 
is  small.  Most  probably  it  is  because  the  Navy  has  not  yet  gained 
a  full  appreciation  of  the  potential  use  and  impact  of  a  Comprehen¬ 
sive  Computerised  Personnel  Management  Data  Base  driven  by  a 
program  suite  with  the  power  of  CODAP. 

2.  The  Navy's  OA  unit  is  in  the  Directorate  of  Naval  Training 
Research  which  is  the  junior  Directorate  of  the  Training  and 
Education  Branch  of  the  Personnel  Division  (Table  1).  Its  total 
strength  is  two  officers  and  two  senior  sailors  with  one  WRAN 
Writer. 


TABLE  1 


CHIEF  OF  NAVAL  PERSONNEL 


Health 

Services 


Manpower 


- 1 - 

Training  and 
Education 


Personal 

Services 


Education 


- , - 

Training 


I 

Training 

Research 


Reserves  and 
Cadets 


OCCUPATIONAL 

ANALYSIS 
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3.  With  a  staff  of  this  size,  the  method  of  work  is  to  proceed 
with  just  one  study  at  a  time  until  the  Data  Base  is  firmly  estab¬ 
lished  on  the  computer.  Subsequently  the  OA  unit  will  continue 

to  provide  an  information  service  from  it  for  as  long  as  the  data 
is  considered  sufficiently  recent  and  pertinent.  Our  method  of 
work  then  is  on  a  one  study,  one  team  basis  and  has  more  in  common 
with  the  approach  of  the  United  States  Marine  Corps  than  with  the 
other  United  States  Services. 

4.  Some  years  ago  at  a  symposium  on  the  "State  of  the  Art  in 
Occupational  Analysis",  Dr  Christal  gave  a  set  of  recommendations 
to  organizations  planning  large  scale  Job  Analysis  using  Task 
Inventories.  One  of  these  was  to  obtain  and  use  the  CODAP  package 
and  to  modify  it  if  necessary  to  meet  individual  needs  -  clearly 

a  recognition  that  the  requirements  of  an  Occupational  Analysis 
and  its  method  of  execution  depend  on  the  size  and  structure  of 
the  personnel  system  which  it  serves.  With  the  range  of  Military 
organizations  now  using  CODAP  for  diverse  reasons,  it  would  indeed 
be  surprising  if  all  user  units  had  the  same  modus  operandi. 

5.  The  United  States  Navy,  for  instance,  directs  its 
Occupational  Analysis  work  towards  Manpower  Planning,  Personnel 
Structures,  Manpower  Modelling  and  Training.  Obviously  it  is  a 
widely  used  activity,  but  bearing  in  mind  the  considerable  research 
investment  which  the  USN  have  made  in  this  field  for  many  years, 
together  with  the  use  of  CODAP  for  the  past  eight  years,  this  is 
hardly  surprising.  It  would  be  a  mistake  however,  if  we  in  the 
RAN  were  to  lavishly  follow  the  methods  used  so  successfully  by  the 
USN. 


6.  Notwithstanding  the  tremendous  difference  in  numerical 

strengths  between  the  USN  and  the  RAN,  the  range  of  ship  types 
operated  and  therefore  the  spectrum  of  work  undertaken,  bears  com¬ 
parison.  To  illustrate  the  possible  differences  in  OA  requirements, 
we  can  compare  the  personnel  frameworks  of  the  RAN  and  the  USN  to 
meet  this  range  of  tasks  (Table  2). 


TABLE  2 

NAVAL  PERSONNEL  SYSTEMS 

R.A.N. 

U.S.N. 

BRANCHES  12 

24  OCCUPATIONAL 

FIELDS 

CATEGORIES  42 

74  RATES 

1200  NAVY  ENLISTED 

CLASSIFICATIONS 
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The  RAN's  "Branch"  is  broadly  similar  to  the  USN's  "Occupational 
Field",  as  is  "Category"  with  the  USN  "Ratings".  The  difference 
must  be  in  range  of  tasks  carried  out  by  the  two  sailors.  How¬ 
ever,  the  USN  breakdown  doesn't  finish  there  -  it  also  lists  1200 
Navy  enlisted  classifications  (NECs)  which  are  important  qualifica¬ 
tions  for  many  postings.  Most  sailors  in  the  USN  have  either  one 
or  two  NECs  to  supplement  this  rating  and  perhaps  to  produce  a  more 
definitive  pattern  of  postings.  Such  a  detailed  structure  can  be 
supported  by  a  half  million  strong  Service,  but  it  would  prove  too 
restrictive  to  the  smaller  Navy. 

7.  Table  3  shows  the  range  of  five  NECs  associated  with  the 

Yeoman  Rate. 


We  can  see  that  the  RAN  Writer  Category  would  cover  much  of  the  work 
of  all  these  rates  -  although  Religious  Program  Specialist  billets 
would  be  difficult  for  him. 

8.  Much  of  the  OA  work  in  the  USN  during  the  early  70s  was  in 
support  of  investigations  into  the  rationalisation  of  career  ladders 
and  NECs,  activities  which  used  task  inventories  covering  the  range 
of  work  of  several  related  NECs. 

9.  With  a  Service  the  size  of  the  RAN,  comprehensive  personnel 
research,  as  carried  out  by  the  US  Military,  is  not  economically 
viable.  For  this  reason,  Job  Analysis  which  started  in  the  RAN  in 
1972,  was  the  result  of  a  perceived  need  within  the  new  training 
system  rather  than  a  thoroughly  researched  support  activity 
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and  consequently  progressed  on  very  much  a  trial  and  error  basis. 
Thus  when  the  CODAP  tool  was  obtained,  its  power  and  width  of 
application  were  not  fully  appreciated.  Consequently  early  studies 
had  limited  aims  since  they  were  initiated  by  individual  Director¬ 
ates  . 

10.  To  date  almost  fifty  percent  of  RAN  personnel  have  each 
completed  one  of  the  seven  questionnaires  so  far  distributed. 

With  the  exception  of  a  small  survey  undertaken  in  support  of  a 
pay  level  investigation,  all  have  been  directed  primarily  at  one 
or  more  stages  of  training  design.  (Table  4) 


TABLE  4 

R.A.N.  OCCUPATIONAL  ANALYSIS  SURVEYS 


CATEGORY/BRANCH 

REASON 

No.  Of  MEMBERS 

Communications 

CODAP  Trial 

700 

Safety  Equipment 

Training 

60 

Marine  (Hull  &  Propulsion) 

Training 

2000 

Weapons  Electrical 

Training 

1600 

Air  Engineering 

Training 

600 

Submarine  Engineering 

Training 

200 

Seaman 

Training 

400 

Stewards 

Pay  Level  Support 

4oo 

6000 

11.  Since  these  surveys  have  been  concerned  with  Job  Analysis, 
the  questionnaires  used  have  been  lengthy  in  terms  of  their  task 
inventories,  yet  the  level  of  task  detail  identified  is  generally 
considered  too  coarse  for  training  design  purposes.  These 
inventories  consist  of  up  to  1 ,000  items,  and  it  is  felt  that 
longer  inventories  would  alienate  both  respondents  and  managers 
to  the  extent  that  the  willing  co-operation  which  the  OA  unit  has 
met  to  date  could  be  lost.  Perhaps  studies  aimed  specifically  at 
training  should  utilise  secondary  questionnaires  restricted  to 
incumbents  of  fairly  narrow  job  definition  such  as  "Leading 
Writer  Pay  Office"  rather  than  "Writer  Category,  All".  This  has 
the  disadvantage  that  since  we  do  not  design  training  modules  for 
such  discrete  employments,  training  design  would  have  to  be  the 
result  of  a  number  of  related  surveys. 
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12.  It  is  anticipated  that  the  Survey  of  Officers  and  Warrant 
Officers  currently  underway  will  have  considerably  more  impact  in 
the  Naval  Personnel  Division  since  many  Directorates  were  actively 
involved  in  structuring  the  several  sections  of  the  Data  Gathering 
Document.  Time  will  tell. 


CURRENT  USE  OF  OA  IH  THE  MILITARY 


Major  A.  Davidson 


1.  As  mentioned  already,  Navy,  Army  and  Air  Force  activities 
and  approaches  to  occupational  analysis  (especially  the  compila¬ 
tion  of  Task  Inventories)  differ  to  a  degree  because  of  the  way 
each  cell  grew  up,  and  the  personnel  structure  within  each  cell. 

2.  MERIT  is  fragmented  in  its  approach  to  the  total  Survey. 
Unlike  the  Navy,  and  to  a  lesser  degree,  unlike  the  Air  Force, 
the  survey  process,  while  controlled  overall  by  myself,  is  split 
into  discrete  areas  of  responsibility.  We  can  divide  these  into 
a  few  basic  groups. 


TABLE  1. 

AREAS  OF  RESPONSIBILITY 


S02 

*  PRIORITIES 

•  contactsaiaison 

•  OVERALL  CONTROL 

S03  (TECH) 

•  BOOKLET  LAYOUT 

*  PROCESSING  METHOD 

•  TECH  LIAISON 

•  FINAL  REPORT 

S03  (ADMIN) 

•  ALL  CORRESPONDENCE 

•  LIAISON/CONTROL  OF  TYPINGARINTING 

JA  (M) 

•  T.I.  COMPILATION 

*  REVIEW 

*  DESPATCH/RECEIPT 

•  EDIT/RECEIPT 
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3.  Apart  from  this  fragmentary  approach,  the  main  point  to 

be  emphasised  is  that  compilation  and  review  of  the  Task  Inventory 
is  carried  out  solely  by  the  Warrant  Officer  to  whom  that  particular 
survey  has  been  allocated.  With  the  assistance  of  the  sponsoring 
directorate  he  organises  both  panels  and  the  venue  and  then  guides 
the  panel  of  job  experts  in  the  Inventory  construction. 


TABLE  2. 

MERIT  -  THE  SURVEY  PROCESS 


S02 

S03 

(TECH) 

S03 

(ADMIN) 

JA 

(M) 

PLANNING 

1,1 

X 

X 

TICC  & 

COMPILATION 

X 

PRINTING 

X 

SURVEY 

X 

RECEIVE/EDIT 

X 

X 

DATA  PREP/ 
PROCESSING 

X 

REPORT 

X 

X 

4.  The  process  shown  in  Table  2  is  comparable  to  the 

procedures  employed  throughout  all  services.  The  pilot  study 
is  an  area  now  being  used  less  and  less  by  Army,  since  the  booklet 
layout  and  administration  procedures  have,  over  the  past  three  or 
so  years,  proved  themselves  adequate  in  allowing  us  to  achieve  our 
objectives . 
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5.  Turning  to  the  actual  survey,  a  point  worth  making  concerns 

the  factors  used  by  Army  respondents  in  rating  the  tasks  performed. 


TABLE  5 
INVOLVEMENT 


Involvement  is  a  measure  of  your  participation  in  the  tasks  you 
perform.  If  you  only  assist  in  a  task  you  will  rate  your 
involvement  as  being  low,  while  tasks  that  you  supervise  will  be 
rated  high.  Select  the  value  which  best  describes  your  participa¬ 
tion  in  the  task. 

1  -  I  ASSIST,  UNDER  SUPERVISION,  IN  THIS  TASK 

2- 1  MOSTLY  ASSIST,  BUT  SOMETIMES  PERFORM  THIS 
TASK 

3- 1  USUALLY  PERFORM,  BUT  SOMETIMES  ASSIST  IN 
THIS  TASK 

4- 1  USUALLY  PERFORM  THIS  TASK 

5- 1  USUALLY  PERFORM,  BUT  SOMETIMES  SUPERVISE 
THIS  TASK 

6- 1  MOSTLY  SUPERVISE,  BUT  SOMETIMES  PERFORM 
THIS  TASK 

7- 1  SUPERVISE  THIS  TASK 


TIME  SPENT 


This  response  (.rating)  is  a  measure  of  the  amount  of  time  you  spend 
on  a  task.  The  rating  you  give  a  task  is  estimated  by  comparing 
the  time  you  spend  on  that  task  with  the  time  on  all  other  tasks. 
Tasks  which  occupy  a  lot  of  your  time  are  rated  high,  while  those 
taking  little  time  are  rated  low.  Try  to  use  the  full  range  of 
the  scale  to  show  the  way  your  work  time  is  distributed. 

1 

2 

3  - 

4  - 

5  - 

6 


WELL  BELOW  AVERAGE 
BELOW  AVERAGE 
SLIGHTLY  BELOW  AVERAGE 
AVERAGE 

SLIGHTLY  ABOVE  AVERAGE 
ABOVE  AVERAGE 


H  AMOUNT  OF  TIME 
SPENT 


7 


WELL  ABOVE  AVERAGE 


101 


6.  The  person  actually  performing  the  job  is  asked  to  rate 
each  task  performed  on  these  two  factors  -  Time  Spent  and  Involve¬ 
ment.  The  Involvement  question  has  the  main  function  of  assisting 
trainers  to  determine  training  requirements  and  levels  of  that 
training  which  proves  necessary. 

7.  The  same  Task  Inventory  is  then  given  to  selected 
supervisors  who  are  asked  to  rate  ALL  tasks  on  one  or  the  other  of 
these  factors. 


TABLE  4 

RESPONSE  GUIDE  -  SUPERVISORS 


IMPORTANCE  RESPONSE 


This  response  (rating;  is  a  measure  of  how  important  the  task  is 
in  relation  to  its  effect  on  the  units  mission  in  peace  or  war. 

If  neglect  or  poor  performance  of  a  task  would  have  little  or  no 
effect  on  the  ability  of  the  unit  to  achieve  its  mission  it  is 
rated  LOW.  A  task,  which  if  neglected  or  performed  poorly, 
would  result  in  rapid  deterioration  of  the  units  ability  to  achieve 
its  mission,  is  rated  HIGH. 

1  -  UNIMPORTANT 

2  -  LOW  IMPORTANCE 

3  -  BELOW  AVERAGE  IMPORTANCE 

4  -  IMPORTANT  (AVERAGE  IMPORTANCE) 

5  -  ABOVE  AVERAGE  IMPORTANCE 

6  -  HIGH  IMPORTANCE 


7 


CRITICAL 
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IMMEDIACY  RESPONSE 


Immediacy  is  an  assessment  of  how  soon  after  taking  up  an  appoint¬ 
ment  (marching  into  the  unit  after  training)  an  incumbent  is 
expected  to  satisfactorily  carry  out  the  task. 

1  -  NOT  WITHIN  TWELVE  (12)  MONTHS 

2  -  WITHIN  TWELVE  (12)  MONTHS 

3  -  WITHIN  SIX  (6)  MONTHS 

4  -  WITHIN  THREE  (3)  MONTHS 

5  -  WITHIN  ONE  ( 1 )  MONTH 

6  -  WITHIN  ONE  ( 1 )  WEEK 

7  -  IMMEDIATELY 

8.  Again  the  training  implication  for  these  factors  is  obvious, 
more  so  in  the  types  of  training  which  should  be  decided  upon. 

-  Formal  classroom  training  on  a  course 

On  the  job  training  while  being  actually 
employed  in  his  trade,  or 

No  training. 

9.  In  the  Army,  an  occupational  analysis  survey  may  be 
initiated  basically  by  one  of  three  methods.  Depending  on  which 
method  is  employed,  the  objectives  vary  to  some  degree. 

a.  Sponsoring  Directorate  Approaches  MERIT.  This 
approach  can  result  in  a  series  of  objectives 
like : 

( 1 )  Should  such  and  such  trade  be  split  into 
two  trades. 

(2)  Should  several  trades  be  amalgamated. 

(3)  What  equipments  are  used  where  and  for  what 
percentage  of  time. 

(4)  Correlation  of  job  satisfaction  with  area/ 
unit  of  posting. 

b.  The  Directorate  of  Personnel  Plans  (DPP)  Approach 
MERIT.  Being  concerned  with  Soldier  Policy,  DPP 
is,  among  many  other  things,  interested  in  Job 
Specifications  being  kept  up  to  date.  That  is, 
that  the  soldier  is  in  fact  doing  what  Army  Office 
thinks  he  is  doing.  The  training  implications 

in  this  area  are  obvious.  DPP  is  responsible  for 
the  Manual  of  Personnel  Administration,  where  also 


-  103  - 


outlined  is  the  MERIT  Charter  which  mentions  that 
most  trades  should  be  surveyed  every  3  to  5  years. 

c.  MERIT  Itself  Initiates.  In  line  with  the  Manual 
of  Personnel  Administration  MERIT  keeps  a  log  of 
all  trades  and  the  dates  they  were  last  surveyed. 

In  discussions  with  the  Sponsoring  directorate, 

MERIT  keeps  a  track  of  introduction  of  new  equipment, 
changes  in  unit  function  or  mission,  variations  in 
training  methods  or  policy  and  trade  structures. 

10.  MERIT  averages  some  15  surveys  a  year.  As  of  last 
August  in  various  stages  of  the  survey  process  we  had  21  surveys. 
Some  of  these  were  carryovers  from  the  last  months  of  1978.  The 
delays  are  mainly  attributed  to  a  somewhat  deficient  mark  sensing 
machine  -  the  OPSCAN.  To  add  to  the  difficulty  the  earliest 
surveys  are  by  far  the  largest  we  have  done.  These  are  mainly  in 
the  clerks  and  storemen  areas. 

11.  Currently  MERIT  is  also  involved  in  carrying  out  a  survey, 
not  really  OA  oriented,  to  validate  the  recommendations  of  the 
Regular  Officer  Development  Committee  (RODC)  to  formalize  Officer 
Employment  Categories  (OEC).  A  broad  ranging  and  technically 
sophisticated  job  analysis  will  be  necessary  in  the  long  run  for 
the  complete  evaluation  and  categorization  of  all  officer  appoint¬ 
ments. 


12.  In  the  short  term,  the  most  important  need  is  to  provide 
adequate  information  on  jobs  to  enable  the  personnel  managers  in 
the  Military  Secretary  Branch  to  manipulate  data  on  individuals 
and  jobs  using  CLIO  (their  computer  facility).  This  will  assist 
in  panelling  officers  for  appointments. 

13.  There  are  other  short  term  needs.  It  is  desired  to  obtain 
information  to  confirm  or  adjust  the  Staff  Officer  Employment 
Categories  proposed  by  the  ROD.C.  This  means  that  the  scope  of  the 
job  analysis  should  be  limited  initially  in  the  non-Corps  area,  as 
this  is  the  area  identified  by  the  RODC  as  most  in  need  of 
attention.  There  is  also  an  urgent  need  to^obtain  better  informa¬ 
tion  on  training  and  education  needs. 
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REPORT  FROM  THE  RAAF 
OCCUPATIONAL  ANALYSIS  CELL 

Squadron  Leader  K.J.  Dowrick,  BSc,  BEd,  MEd  (Admin) 

INTRODUCTION 


1 .  The  overall  aim  of  this  paper  is  to  provide  a  status 
report  from  the  RAAF  occupational  analysis  (OA)  cell.  The  cell 
is  known  as  GT3A  OA  and  Training  Evaluation  Section;  it  is 
located  in  the  Ground  Training  Division,  Headquarters  Support 
Command  (HQSC) ,  Victoria  Barracks,  Melbourne.  GT3A  Section  is 
responsible  for:  the  development  of  OA  and  training  evaluation 
procedures  applicable  to  the  RAAF;  the  conduct  of  RAAF  OA  and 
other  surveys;  and  the  training  of  RAAF  managers  in  the  use  of 
OA  data.  The  Section  is  established  for  three  officers  (GT3A 
Squadron  Leader  of  the  Special  Duties  Education  Category  (ED), 

GT3A1  Flight  Lieutenant  (FLTLT)  ED  and  GT3A2  (FLTLT  ED))  and  two 
supporting  clerical  staff.  Over  the  past  six  years  the  section 
has  been  fortunate  in  having  the  GT3A1  post  manned  by  a  USAF 
exchange  officer.  Each  of  these  exchange  officers  have  made 
many  valuable  contributions  to  the  RAAF  OA  programme.  Currently, 
the  post  is  manned  by  Captain  C.D.  Gorman  (USAF). 

2.  HQSC  (Ground  Training  Division)  is  tasked  by  Department 
of  Defence  Air  Force  (DEFAIR)  to  undertake  OA  surveys  on  selected 
musterings.  The  tasking  for  1978/79  is  shown  at  Table  1. 

Ideally,  these  surveys  should  be  updated  every  three  to  four 
years  so  reasonably  current  OA  data  is  available  on  each  of  the 
108  different  musterings.  However,  this  would  require  GT3A 

to  survey  at  least  27  musterings  per  year.  Over  the  past  three 
years,  the  GT3A  Section  has  managed  to  survey  only  seven  musterings 
per  year.  Thus,  with  such  a  shortfall  from  the  desired  output, 
the  sections  tasking  has  been  focused  on  surveying  those  musterings 
where  it  is  felt  the  need  is  greatest. 


TABLE  1 

OCCUPATIONAL  ANALYSIS  AND  TRAINING 
EVALUATION  SECTION  (GT3A) 

PROJECTS  1978/79 

Pro.iect 


CLK,  CLKA,  CLKG,  CLKM 

OCCUPATIONAL 

SURVEY 

PHOTO 

OCCUPATIONAL 

SURVEY 

MEDORD 

OCCUPATIONAL 

SURVEY 

INSTFITT 

OCCUPATIONAL 

SURVEY 

AFFITT 

OCCUPATIONAL 

SURVEY 

AMWKR 

OCCUPATIONAL 

SURVEY 

SURFIN 

OCCUPATIONAL 

SURVEY 

SEW 

OCCUPATIONAL 

SURVEY 

RADTECHA,  RADTECHG ,  TELSTECH 

OCCUPATIONAL 

SURVEY 

B01LATT 

OCCUPATIONAL 

SURVEY 

SI,  3GUARD 

OCCUPATIONAL 

SURVEY 
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Project 

COOK 

EDASST 

DMANENG 

LINESMAN 

PTI 

HYGIENE  INSP 

ELECTRONIC  PRINCIPLES  (RADIO) 

ENGINE  FITTER  COURSE  VALIDATION 

EQUIPMENT  OFFICERS  INITIAL  COURSE  STUDY 

VALIDATION  OF  THE  MATHEMATICS  CONTENT 
OF  AIRMOVEMENT  COURSE 


OCCUPATIONAL  SURVEY 
OCCUPATIONAL  SURVEY 
OCCUPATIONAL  SURVEY 
OCCUPATIONAL  SURVEY 
OCCUPATIONAL  SURVEY 
OCCUPATIONAL  SURVEY 
SPECIAL  PROJECT 
SPECIAL  PROJECT 
SPECIAL  PROJECT 

SPECIAL  PROJECT 


AIM 


3.  As  mentioned  above,  this  brief  provides  an  outline  of 

the  RAAF  OA  programme.  More  specifically  the  aim  of  this  brief 
is  to: 

a.  Discuss  in  general  terms  the  kinds  of  OA 
programmes  being  conducted. 

b.  Discuss  the  uses  to  which  OA  data  has  been 
put. 

c.  Discuss  the  future  of  the  RAAF  OA  programme. 


KINDS  OF  PROGRAMMES  BEING  CONDUCTED 


Description  of  RAAF  OA 


4.  In  order  to  give  you  an  idea  of  the  type  of  data  we 
collect,  I  will  briefly  describe  the  response  booklets  used  by 
the  RAAF.  Each  booklet  contains  two  sections.  The  first 
section  consists  of  background  questions  to  be  answered  by  the 
respondent  about  his  job  and  himself  ...  time  on  the  job,  tools 
utilized,  equipment  worked  on,  interest  in  the  job  and  so  on. 

Any  other  questions  of  interest  can  be  included  in  the  background 
information  section  of  an  OA  booklet. 

5.  The  second  section  of  the  booklet  is  simply  a  list  of 
all  the  significant  tasks  performed  by  members  of  the  mustering 
being  surveyed.  That  is,  it  includes  tasks  being  performed  by 
all  rank3  in  the  mustering.  Provided  this  task  list  is  properly 
constructed,  then  every  member  of  the  mustering  should  be  able  to 
define  his  job  adequately  in  terms  of  a  subset  of  tasks  in  the 
mustering.  Table  2  lists  the  procedure  used  by  the  RAAF  OA  cell 
for  the  development  of  the  task  inventory  and  background  information 
sections. 
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TABLE  2 

PROCEDURE  FOR  THE  DEVELOPMENT  OF 
A  TASK  INVENTORY 


1 .  Preliminary  preparation  by  GT3A. 

2.  Review  by  specialist  NCOs. 

3.  Review  by  command  staff  officers. 

4.  Field  review  by  specialist  NCOs  on  units. 

5.  Construction  of  final  inventory. 

6.  Normally,  the  RAAF  aims  to  survey  between  80  and  100 
percent  of  the  members  in  the  mustering.  Presidents  of  Unit 
Trade  Test  Boards  (UTTB)  are  responsible  for  the  administration 
of  the  survey  at  unit  level  and  at  units  where  there  is  no  UTTB, 
commanding  officers  are  responsible.  To  be  eligible  to  complete 
the  survey,  a  respondent  must  have  been  in  his  present  mustering 
for  at  least  two  months  and  in  his  present  posting  and  present  job 
for  at  least  one  month. 

7.  Each  member  selected  to  complete  the  survey  checks  the 
tasks  he  performed  and  how  his  worktime  is  distributed  across  the 
tasks  in  his  job.  At  a  later  date  about  45  supervisors  rate 
selected  tasks  on  one  of  the  following  factors: 

task  learning  difficulty,  consequence  of  inadequate 
performance,  and  need  for  immediate  performance. 

8.  The  responses  are  then  processed  (either  by  keypunching 
on  to  cards  or  optical  scanning  onto  tape)  to  produce  an  input 
suitable  for  the  computer  to  enable  the  data  bank  to  be  established. 
The  basic  outputs  from  the  data  banks  are  job  specifications; 
these  list  all  the  tasks  performed  by  the  sample  or  part  of  it  and 
are  arranged  in  order  of  time  spent  on  or  percentage  performing 
these  tasks.  From  selection  of  such  specifications  is  derived 

the  whole  family  of  comparisons,  matrices  and  diagrams. 

9.  At  this  point  I  feel  it  is  appropriate  to  state  an 
important  point  which  is  often  overlooked.  OA  produces  an  accurate 
picture  of  the  present  state,  this  is  an  important  point  to 
recognize  since  OA  does  not  forecast  what  the  future  employment 
will  be,  nor  does  it  forecast  what  it  could  be.  This  must  be 

left  to  policy  makers,  administrators,  or  trainers,  though  they 
should  be  guided  by  the  existing  situation  as  shown  by  the  OA 
data. 


10.  The  initial  request  from  our  OA  cell  to  Computer  Services 

Division  (CSD)  Canberra  lists  the  CODAP  outputs  such  as  DIAGRAM, 
PRINT? AR,  and  some  initial  job  descriptions  for  ranks.  Examination 
and  interpretation  of  these  outputs  generates  further  requests 
for  addition  CODAP  outputs  eg,  job  description  or  AUTOJT  to  help 
identify  different  groups  on  the  DIAGRAM. 


#V-  llflra 
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11.  After  the  analysis  of  the  data  is  completed,  the 

occupational  analyst  involved  in  surveying  the  mustering  writes 
a  report  summarizing  aspects  he  believes  are  relevant.  After 
the  report  is  printed,  a  briefing  is  given  at  DEFAIR  to 
representatives  of  the  Directorate  of  Training,  the  specialist 
officer  for  the  mustering  and  other  interested  members.  Table  3 
summarizes  the  various  phases  of  an  0A  survey  on  a  mustering. 


TABLE  3 

VARIOUS  PHASES  OF  OCCUPATIONAL 
ANALYSIS  SURVEY 


+  BACKGROUND 
INFORMATION 


Special  Projects 

12.  As  well  as  tasking  in  OA  projects  the  OA  and  Training 

Evaluation  Section  also  is  tasked  with  a  number  of  special 
projects  which  require  the  development  of  a  questionnaire  to 
suit  the  requirements  of  each  project.  The  following  special 
projects  were  undertaken  in  the  1978/79  period:  Electronic 
Principles  Inventory  (EPI),  Engine  Fitter  Course  Validation, 
Survey  of  the  Contents  of  the  Equipment  Officers'  Initial  Course 
(EOIC; ,  Validation  of  the  Mathematical  Contents  of  the  Air- 
movements  Course. 
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13.  The  EPI  survey  involved  validating  the  electronics 
content  of  the  trade  training  courses  for  the  mechanical  and 
electrical  trades.  So  informative  was  the  data  from  this 
project,  that  DEFAIR  requested  a  similar  survey  be  conducted  to 
validate  the  electronics  content  of  the  radio  trades  basic 
trade  training  course.  The  EPI  survey  was  outlined  by  Captain 
Gorman  in  his  lecture  yesterday. 

14.  The  EOIC  study's  aim  was  to  collect  data  to  validate 

the  course  and  assist  in  any  course  redesign  that  may  be  considered 
necessary.  The  following  information  was  received  concerning 
152  syllabus  items:  time  spent;  use  in  job;  frequency  of  use; 
level  of  training,  and  training  received.  The  survey  provided 
valuable  information  for  improving  the  course. 

15.  The  remaining  two  special  projects  did' not  employ 
CODAP  outputs  and  therefore  will  not  be  outlined  in  this  brief. 
However,  CODAP  was  used  to  provide  outputs  in  the  Officers 
Education  and  Training  (OET)  Project.  This  project  was  outlined 
by  Wing  Commander  Huet  in  his  lecture  yesterday. 


Use  of  OA  Data 


16.  GT3A  Section  has  recently  developed  a  procedure  for 
using  OA  in  curriculum  design.  This  procedure  has  been  success¬ 
fully  used  and  it  appears  that  it  is  in  this  area  that  RAAF  OA 
could  make  one  of  the  most  valuable  contributions  to  training. 

In  addition,  OA  data  has  been  used  for  some  time  to  assist  in 
the  production  of  trade  standards  and  trade  tests.  The  procedures 
used  in  curriculum  design  and  trade  tests  will  be  outlined  in 
the  session  titled  "Production  and  Validation  of  Trade  Specifica¬ 
tions  and  Graduation  Requirements". 

17.  As  well  as  the  special  projects  designed  to  validate 
part  or  all  of  a  training  course,  validation  of  some  courses 

has  been  achieved  using  OA  data.  OA  data  can  provide  information 
which  can  identify  tasks  being  trained  which  incumbents  are 
unlikely  to  encounter  on  the  job.  Although  schools  can  usually 
determine  where  additional  training  should  be  given  through 
feedback  from  user  units,  it  is  far  more  difficult  for  them  to 
identify  training  that  can  be  safely  eliminated  or  reduced. 

Thus,  OA  can  make  additional  contributions  to  validation  of 
training. 

18.  Finally,  the  potential  for  use  of  OA  data  in  other 
areas  is  considerable.  In  the  future  survey  data  may  be  used 
to  assist  in  the  restructering  of  musterings  or  determining 
their  responsibilities;  in  posting  actions  which  may  match 
specific  jobs  with  specific  individuals  known  (through  the  OA 
cell)  to  have  the  necessary  prior  experience  for  the  job  in 
question;  and  in  answering  other  personnel/administration 
questions. 


Recent  Changes  in  RAA F  OA 


19»  One  of  the  major  changes  made  in  the  collection  of  OA 

data  was  the  removal  of  the  involvement  scale  from  the  task 
inventory,  leaving  only  the  time  spent  scale.  This  decision 
was  mainly  based  on  practically  no  user  requirement  for  involvement 
data  and  the  ease  of  collecting  this  information  on  specific 
tasks  (if  required).  The  decision  to  discontinue  using  the 
involvement  scale  resulted  in  a  reduction  of  almost  50  percent 
in  the  number  of  OPSCAN  sheets  required  for  each  survey. 

20.  Another  change  of  interest  is  the  introduction  of  the 

matrix  format  for  some  musterings  that  encompass  a  particularly 
large  number  of  tasks.  Attempts  to  survey  such  broad  musterings 
may  result  in  a  linear  list  of  tasks  which  is  probably  more  than 
a  respondent  can  handle  (without  a  probable  reduction  in  validity 
and  reliability).  The  question  of  reliability  and  validity  will 
be  researched,  in  the  near  future  by  comparing  the  data  from 
the  matrix  format  with  that  from  the  well  researched  linear 
f  ormat . 


The  Future  of  the  RAAF  OA  Programme 

21  .  We  in  the  RAAF  OA  cell  are  convinced  that  the  CODAP  has 

much  to  offer  the  RAAF  but,  unfortunately ,  only  limited  use 
seems  to  be  being  made  of  the  data  since  CODAP  became  available 
to  the  RAAF  in  late  1974.  This  can  partly  attributed  to  the 
fact  that  no  individual  or  group  was  specifically  tasked  with 
reviewing  training  using  the  available  OA  data  as  an  input. 

Thus,  it  is  up  to  the  individual  specialist  officer  to  conduct 
such  a  review  if  he  can  somehow  make  the  time  available  from 
his  primary  duties  to  do  so.  In  most  cases  initial  interest 
is  shown  by  the  specialist  officer  in  the  results  of  the  survey, 
but  usually  the  OA  data  ends  up  on  a  shelf  gathering  dust. 

Thus,  unless  a  team  is  established  for  the  specific  purpose  of 
reviewing  training,  there  is  little  likelyhood  of  the  OA  data 
being  used  to  any  great  extent. 

22.  The  RAAF  OA  cell  also  sees  the  lack  of  use  of  data  as 

being  partly  attributed  to  its  failure  to  convince  the  potential 
user  of  the  advantage  of  using  the  OA  data.  Thus,  to  overcome 
our  failure  to  sell  the  programme,  we  have  undertaken  a  public 
relations  programme.  This  includes  personally  contacting 
potential  users  for  their  inputs  to  a  survey  instrument  and 
personally  delivering  (complete  with  briefing  and  discussion 
session)  every  OA  report  to  those  responsible  for  the  training 
in  DEFAIR.  Our  PR  efforts  have  had  some  productive  effects  as 
shown  by  the  already  developing  interest  in  the  RAAF  OA  programme. 
However,  our  public  relations  programme  needs  to  be  extended  if 
we  are  to  convince  other  potential  users  throughout  the  RAAF  of 
what  the  OA  programmes  are  capable  of  doing.  The  areas  to  which 
such  PR  can  be  extended  includes: 

a.  conducting  workshops  on  how  to  use  OA  data, 

b.  conducting  lectures  on  OA  for  those  who 
administer  surveys,  and 
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c.  visiting  units  for  discussion  on  and  inputs 
to  0A  surveys. 

To  accomplish  this  PR,  some  allowance  will  need  to  be  made  in 
the  survey  tasking  for  such  activities.  Currently,  almost  all 
the  available  time  of  the  members  in  the  RAAP  OA  cell  is  spent 
in  accomplishing  our  tasking.  If  fewer  surveys  are  undertaken, 
more  time  can  be  devoted  to  attempting  to  have  the  data  from 
these  surveys  used. 


CONCLUSION 

23.  OA  is  making  contributions  to  the  production  of  trade 

standards  and  course  design,  and  the  validation  of  training. 
However,  progress  in  using  OA  data  has  been  slow.  If  OA  is  to 
achieve  its  full  potential  for  use  in  the  RAAP,  a  greater  effort 
will  need  to  be  applied  to  convincing  the  potential  users  of  the 
benefits  that  can  be  gained  from  its  use. 


110  - 


3 


c.  visiting  units  for  discussion  on  and  inputs 
to  OA  surveys. 

To  accomplish  this  PR,  some  allowance  will  need  to  be  made  in 
the  survey  tasking  for  such  activities.  Currently,  almost  all 
the  available  time  of  the  members  in  the  RAAP  OA  cell  is  spent 
in  accomplishing  our  tasking.  If  fewer  surveys  are  undertaken, 
more  time  can  be  devoted  to  attempting  to  have  the  data  from 
these  surveys  used. 


CONCLUSION 


23.  OA  is  making  contributions  to  the  production  of  trade 

standards  and  course  design,  and  the  validation  of  training. 
However,  progress  in  using  OA  data  has  been  slow.  If  OA  is  to 
achieve  its  full  potential  for  use  in  the  RAAP,  a  greater  effort 
will  need  to  be  applied  to  convincing  the  potential  users  of  the 
benefits  that  can  be  gained  from  its  use. 


REPORT  FROM  SPECIAL  STUDIES  SECTION 


Wing  Commander  N.C.  Bleakley,  BSc,  Dip  Ed 


1 o  Special  Studies  Section  in  Establishments  Division  Central 

Office  is  established  for  three  officers,  one  from  each  Service,  and 
is  currently  responsible  to  the  Director,  Services  Establishment 
Support.  Until  recently,  Special  Studies  enjoyed  Directorate 
status;  previous  to  that,  the  Directorate  was  known  as  the 
Directorate  of  Job  Evaluation. 

2.  The  issue  of  joint-Service  activity  in  occupational  analysis 
was  probably  addressed  for  the  first  time  in  1969  when  the  Personnel 
Administrative  Officers'  Committee  made  recommendations  to  this 
effect.  The  impetus  at  that  time  was  the  need  for  a  better 
knowledge  of  job  content  for  job  evaluation  studies  which  were 
intended  to  lead  to  a  pay  system  that  rewarded  the  military  skills 

of  Service  occupations. 

3.  Co-ordination  of  OA  activities  was  temporarily  halted  when 
the  Kerr-Woodward  Committee  was  established  to  enquire  into  salary 
and  wage  systems  of  the  Services.  However,  as  a  result  of  the 
Committee's  recommendations,  a  full-scale  job  evaluation  study  of 
officers  and  other  ranks  was  undertaken. 

4.  This  job  evaluation  exercise  was  carried  out  by  the 
Directorate  of  Job  Evaluation  (DJE)  during  the  period  1974-1976. 

The  method  employed  was  known  as  the  Hay  System,  as  it  was 
developed  by  Hay  Associates  (Australasia)  ltd.  I  want  to  present 
to  you  a  very  brief  outline  of  the  principles  involved  in  the  Hay 
System  since  it  is  relevant  to  a  session  being  conducted  tomorrow. 

5.  The  Hay  System  measures  each  job  in  terms  of  three 
factors : 

a.  know-how, 

b.  problem-solving,  and 

c.  accountability. 

6.  Hay  Associates  claim  these  factors  are  common  in  varying 
degrees  to  all  types  of  jobs  and,  when  measured  collectively,  are 
a  fair  expression  of  relative  job  content.  The  three  factors  are 
defined  briefly  as: 

a.  Enow-How  is  the  sum  total,  however  acquired,  of  every 
kind  of  knowledge  and  skill  required  for  competent 
job  performance. 

b.  Problem-Solving  is  the  thinking  requirement  in  a  job 
for  analyzing,  evaluating,  creating,  reasoning, 
arriving  at  and  making  conclusions. 

c.  Accountability  is  the  answerability  for  action  and 
for  consequences  of  that  action. 
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7.  Using  a  guide  chart  and  profiles,  each  job  is  given  a 
score  which  is  the  sum  of  each  of  the  factor  scores.  The  totality 
of  all  the  job  scores  forms  a  job  evaluation  benchmark  scale. 

8.  Whilst  Special  Studies  Section  still  has  a  responsibility 
to  carry  out  position  scoring  according  to  the  Hay  System,  as 
required,  it  is  only  an  infrequent  task  now  since  the  Service 
Offices  have  expressed  dissatisfaction  with  the  method,  especially 
in  its  application  to  Other  Rank  categories. 

9.  Defence  Central  first  became  closely  involved  with  CODAP 
when  a  pilot  study  was  carried  out  by  DJE  in  the  Communication 
Operatives  categories  using  CODAP  techniques.  This  pilot  study 
was  proposed  in  Sep  74  and  there  seems  to  have  been  some  change 
in  the  aims  between  that  time  and  when  the  report  was  presented 
in  May  1977.  The  original  aim  was  to  assess  the  validity  and 
usefulness  of  a  single  computerised  OA  system  for  the  Australian 
Defence  Force.  However,  in  Apr  75 »  the  aim  of  the  project  had 
been  changed  'to  determine  procedures  to  be  adopted  in  future 
joint-Service  and  single-Service  OA  projects'  which  is  rather  a 
different  kettle  of  fish. 

10.  The  details  of  the  COMMSOP  study  are  not  important  here. 
Its  significant  lies  in  the  fact  that  several  very  important  issues 
were  addressed  for  the  first  time: 

a.  the  problems  of  delineation  of  responsibilities 
between  Central  Office,  the  single  Services  and 
Computing  Services  Division; 

b.  the  avenues  for  liaison; 

c.  processing  delays; 

d.  the  problems  of  administering  a  tri-Service  study 
when  the  three  Service  OA  sections  were  so 
geographically  separated,  especially  with  RAAF  in 
Melbourne;  and 

e.  the  problems  of  lack  of  expertise. 

1 1  .  During  the  COMMSOP  study  a  CODAP  Interface  Officer  was 

established  in  CSD  in  order  to  solve  some  of  these  problems. 

Whilst  this  was  a  most  worthwhile  achievement,  many  of  the 
problems  are  still  evident,  living  outside  his  control.  Special 
Studies  Section  has  a  responsibility  to  try  to  solve  them. 


THE  APPLICATION  OF  OCCUPATIONAL 
ANALYSIS  FOR  DETERMINING 
OFFICER  RANKS 

By  Dr  R.E.  Christal 


1  .  Job  evaluation  is  a  tough  area  of  research  because  it 

is  so  politically  sensitive.  No  matter  what  you  come  out 
with,  people  suspect  that  there  has  been  hanky-panky  somewhere 
along  the  line.  However,  the  study  I  shall  describe  is  well 
documented,  and  we  can  defend  it  in  a  number  of  ways. 

2.  We  were  asked  by  the  Director  of  Manpower  to  determine 
appropriate  grade  distributions  for  all  officer  utilization 
fields  in  the  Air  Force.  That  is  a  challenging  job  because 
you  cannot  go  to  the  hardware  store  and  buy  a  yearstick  that 
measures  grades  and  lay  it  alongside  all  the  jobs.  Neverthe¬ 
less,  we  suggested  to  the  Director  of  Manpower  that,  if  he 
could  tell  us  the  appropriate  grade  levels  for  a  sample  of 
jobs,  we  could  use  a  policy  capturing  model  to  determine  the 
appropriate  distribution  for  the  entire  force.  The  first 
important  step  was  to  get  a  sample  of  job  descriptions  with 
which  to  build  a  yardstick. 

3.  At  that  time,  we  did  not  have  CODAP,  and  we  did  not 
have  job  descriptions  for  every  officer  in  the  Air  Force.  So 
we  had  to  develop  job  descriptions  for  the  study.  The  form 
for  the  description  was  relatively  simple.  There  was  a  block 
for  job  title,  location,  and  organization.  Then  there  was  a 
section  where  officers  wrote  a  narrative  on  the  duties  and 
tasks  performed.  In  a  special  comments  section,  an  officer 
could  write  in  things  that  might  be  outstanding  about  his  job, 
but  which  could  not  be  inferred  from  the  narrative.  Additional 
information  was  provided  on  the  present  grade  authorization, 

the  grade  of  the  supervisor,  and  the  educational  level  of  the 
incumbent.  Then  the  supervisor  of  the  officer  was  required 
to  read  the  description  and  certify  that  it  was  accurate;  he 
was  asked  also  to  give  his  judgment  as  to  the  appropriate  grade 
level  for  that  position.  We  gathered  a  total  of  80,000  job 
descriptions  and  then  selected  a  sample  of  3>575.  We  took 
two  descriptions  from  each  authorized  rank  within  each  specialty 
that  had  six  or  more  officers  assigned.  Then,  for  specialties 
where  large  numbers  of  officers  were  involved,  or  where  the 
jobs  were  very  heterogeneous,  we  added  more  job  descriptions 
to  ensure  that  the  stratified  sample  was  completely  represent¬ 
ative  of  the  Air  Force. 

4.  The  next  important  step  was  to  ensure  that  the  proced¬ 
ures  for  determining  grade  levels  were  publicly  defensible. 

The  Director  of  Manpower  believed  that  the  only  people  who 
knew  the  jobs  in  the  Air  Force  were  the  officers  who  worked  in 
that  service.  So  he  selected  a  Board  of  22  colonels  who  were 
to  make  judgments  about  appropriate  grade  levels  for  those 

3 i 575  jobs.  He  wrote  to  the  major  commands  and  gave  the  spec¬ 
ifications  for  the  colonels  he  wanted  for  a  specia1  assignment 
at  the  Pentagon.  The  specifications  were  so  written  that  the 
members  of  the  Board  collectively  had  had  experience  on  almost 
every  type  of  job  in  the  Air  Force.  The  chairman  of  the  Board 
was  a  newly  retired  three-star  general.  Board  members  were 
instructed  by  the  Director  to  take  off  their  command  hats  and 
to  make  truthful  and  unbiased  judgments.  Members  were  required 
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to  read  and  understand  each  job  description  before  they  made 
their  ratings.  If  someone  was  not  sure  what  was  involved  in 
a  job,  he  could  talk  to  a  member  who  had  had  experience  in  that 
domain.  Also,  there  were  people  standing  by  at  the  Pentagon 
who  were  experts  in  the  various  discipline  areas  of  science  and 
engineering.  The  telephone  numbers  of  the  incumbents  and 
their  supervisors  were  available  also  so  that  members  could 
talk  to  the  actual  men  on  the  job.  Civilian  scientist- 
observers  were  used  to  control  the  rating  board  and  make  sure 
the  ground  rules  were  followed. 

5.  For  each  job  description,  we  obtained  five  independent 
judgments.  The  job  descriptions  were  placed  in  booklets  in 
groups  of  twenty-five.  Members  were  arranged  so  that  no  two 
officers  were  rating  the  same  job  at  the  same  time.  No  two 
officers  were  allowed  to  sit  next  to  each  other  for  more  than 
an  hour.  Everything  was  being  observed  to  make  sure  that 
the  judgments  were  perfectly  independent. 

6.  The  grade  rating  scale  had  16  levels  as  follows: 

General  1  level 

Colonel  3  levels 

Lieutenant  Colonel  3  levels 

Major  3  levels 

Captain  3  levels 

All  Lieutenants  3  levels 

The  three  levels  for  colonel  were  for  a  junior  level,  a  senior 
level,  and  a  level  of  about  average.  The  same  applied  for  the 
other  ranks  except  for  general. 

7.  We  were  prepared  to  abort  the  entire  study  if  we  could 
not  defend  the  ratings  of  the  Board  on  the  subset  of  3 >575 
jobs.  We  did  some  simple  analyses  to  find  out  if  the  raters 
had  actually  agreed  on  the  grade  levels  for  those  jobs.  The 
inter-rater  agreement  co-efficient  was  0.92.  This  was  higher 
than  expected, and  suggested  that  the  judges  were  looking  at 
the  jobs  in  the  same  manner.  It  suggested,  also,  that  if  we 
were  to  repeat  the  study  with  an  independent  board  selected 

in  the  same  way,  we  would  expect  to  get  a  high  correlation 
between  the  two  sets  of  ratings.  We  had  asked  the  judges  to 
express  their  feeling  of  confidence  about  the  grades  they  were 
assigning  on  a  three-point  scale.  For  2,387  positions  at  least 
four  out  of  five  judges  reported  the  highest  level  of  confidence 
in  their  ratings.  Then  we  investigated  the  sorts  of  biases 
the  various  officers  revealed  in  their  ratings.  We  could  not 
find  a  single  rater  or  a  single  category  that  we  could  discard 
because  of  preferential  bias  towards  a  class  of  job.  Another 
thing  we  found  was  that  there  was  no  tendency  towards  a  general 
inflation  of  grade;  the  judges  said  that  some  jobs  were  graded 
too  high  and  some  were  graded  too  low.  In  fact,  the  Board 
claimed  that  almost  half  of  the  jobs  that  were  in  the  sample  of 
3 >575  were  not  graded  correctly.  At  this  point,  we  felt 
confident  that  there  must  be  some  meaning  to  grade  and  that  all 
the  judges  had  somewhat  the  same  concept.  All  things  consider¬ 
ed,  we  decided  that  we  could  accept  the  Board  members'  ratings 
as  criteria. 


8.  Obviously,  the  Board  could  not  rate  100,000  Air  Force 

jobs,  so  we  had  to  try  another  approach.  Our  goal  was  to 
determine  the  factors  that  the  judges  considered  in  making 
their  ratings,  then  weigh  their  factors  together  to  assign  the 
same  grade  levels  to  jobs  as  the  Board.  That  is  a  policy 
capturing  model.  First,  we  considered  all  the  possible  job 
requirement  factors  that  may  have  entered  into  the  deliberations 
eg,  management,  communication  skills,  judgment,  and  so  on. 
Historically,  these  factors  have  been  used  in  many  executive 
evaluation  programs.  Then  we  added  such  factors  as  the 
position  the  job  occupied  in  the  organizational  structure. 
Altogether,  we  generated  about  200  different  predictors.  Then 
we  obtained  ratings  on  the  factors  from  independent  groups  of 
lieutenant  colonels  and  majors,  and  computed  an  equation  which 
would  predict  the  grades  assigned.  We  simplified  the  equation 
by  minimizing  the  number  of  factors  and  maximizing  its  face 
validity.  This  made  the  equation  more  easily  understandable. 

At  this  point,  I  shall  define  several  of  the  factors.  For 
example,  management  had  to  do  with  the  level,  the  variety,  and 
the  complexity  of  the  management  controls.  Planning  involved 
the  scope  and  the  significance  of  what  was  being  planned  as 
well  as  the  time-span  involved.  Special  training  and  work 
experience  were  not  concerned  with  professional  military 
education  but  with  experience  that  had  to  be  obtained  before 
a  person  could  perform  a  specific  job  adequately.  Judgment 
and  decision  making  had  to  do  with  the  importance  and  the 
independence  of  the  decisions,  the  likely  impact  of  those  dec¬ 
isions.  Communication  skills  involved  the  level  of  the  person 
or  agency  involved  and  the  impact  that  poor  communication 
might  have  on  the  organization.  The  final  equation  contained 


only  nine  factors,  which  are  listed  in  Table  1.  The  first 
seven  factors  are  the  primary  definers  of  grade. 

Variable  Validity 

Management  . 75 

Planning  . 7 1 

Special  Training  &  Work  Experiences  .45 

Judgment  &  Decision-making  .60 

Communication  Skills  .65 

Level  of  Organization  .55 

Level  Within  Organization  .55 

Mean  Grade  Rating  by  Field  Judges  .89 

Supervisors  Judgment  .65 

.92 


Final  Equation 

Table  1 .  Final  Policy  Equation 
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The  equation,  had  a  validity  of  0.92.  Eighty- two  percent  of 

all  jobs  in  the  sample  were  pin-pointed  to  a  third  of  a  grade 
with  the  equation.  Errors  were  randomly  distributed  across 
pilots,  engineers,  scientists,  and  so  on.  So  although  we 
though  we  might  have  to  develop  one  equation  for  science  and 
engineering  and  another  for  pilots,  it  did  not  turn  out  that 
way.  Evidently,  in  the  minds  of  Air  Force  officers,  grade 
has  a  universal  meaning. 

9.  After  establishing  the  equation,  we  gathered  data  on 
10,000  more  jobs.  Then  we  applied  the  equation  to  determine 
the  appropriate  grade  level  for  each  job.  At  this  point,  it 
was  obvious  that  the  Air  Force  could  not  implement  the  results 
without  some  compromise.  There  are  two  ways  to  determine 
grade  authorizations.  You  can  support  an  orderly  career 
progression  so  that  every  officer  has  some  opportunity  of  being 
promoted  at  the  various  career  points.  This  requires  a  model 
showing  what  the  distribution  of  grades  has  to  be  in  order  to 
support  the  careers'  plan  for  the  whole  force.  But  what  we 
are  discussing  are  the  appropriate  grade  levels  for  jobs  based 
upon  job  content  and  responsibility  levels.  The  question  is: 
do  you  want  to  pay  a  colonel  to  do  a  job  that  a  major  could  do? 

10.  Now,  I  shall  discuss  some  of  the  findings  at  random. 

We  found  that  we  had  too  many  lieutenant  colonels.  There 
were  3,156  pilots  and  navigators  in  flying  jobs.  If  we  had 
implemented  the  Board's  modifications,  2,346  of  those  lieuten¬ 
ant  colonel  jobs  would  have  been  downgraded  to  major  or  below. 
You  do  not  need  a  lieutenant  colonel  to  fly  a  plane.  On  the 
other  hand,  in  the  operations  area,  many  jobs  were  grossly 
undergraded.  Instead  of  135  lieutenant  colonel  jobs,  there 
should  have  been  413*  This  indicated  that  there  were  a  lot 
of  majors  and  captains  in  Air  Force  operations  who  have  very 
responsible  jobs.  Finally,  in  the  science  and  engineering 
areas,  there  would  have  been  a  gross  upgrading  of  jobs.  Now, 

I  shall  mention  one  of  the  payoffs  based  upon  the  study.  The 
Air  Force  was  able  to  get  Defense  Department  approval  for 
2,000  more  authorizations  at  the  major  level.  This  helped 

overcome  a  promotion  barrier  for  2,000  captians  who  would  not 
have  been  promoted  otherwise. 

1 1 o  Since  the  completion  of  the  study,  we  have  developed 

benchmark  scales  with  job  titles  on  each  factor.  Table  2 
shows  part  of  the  benchmark  scale  for  the  management  area. 

Management  The  level  of  executive,  and  managerial 
skills  required  in  the  job.  Consider 
the  complexity,  variety,  and  level  of 
the  activities  which  are  directed,  organ¬ 
ized,  co-ordinated,  controlled,  commanded, 
or  evaluated . 

LEVEL  9:  Director  of  Budget,  HQ  Major  Air  Command 

Commander,  Combat  Support  Gp  (Overseas) 

Wing  Commander,  Tactical  Control  Wg  (Overseas) 


□ 
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LEVEL  5:  Traffic  Management  Officer,  Transportation  Sq 

Base  Communications  Maintenance  Officer, 
Communications  Sq 

Missile  Combat  Crew  Commander,  Strategic  Missile 
Sq 

LEVEL  1 :  Clinical  Psychologist,  USAF  Hospital 

Psychiatric  Social  Worker,  USAF  Hospital 
Helicopter  Pilot  Single  Rotor,  Air  Base  Sq 

Table  2.  Management  Area  Benchmark  Scale 

This  shows  the  top  level  (9)  the  middle  level  (5)  and  the  bottom 
level  (1).  It  will  be  noted  that  helicopter  pilot  is  at  the 
bottom  on  management,  but  on  other  factors,  this  job  would  be 
near  the  top. 

12.  To  summarize ,  I  think  that  this  was  an  honest  and  a 
valuable  study.  It  gave  the  Air  Force  a  clearer  concept  of 
grade,  and  defined  job  requirements  based  on  job  content  and 
responsibility  level. 
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PRODUCTION  AND  VALIDATION  OF  TRADE 
STANDARDS  AND  COURSE  TERMINAL  OBJECTIVES 


Squadron  Leader  K.J.  Dowrick 
Royal  Australia  Air  Force 


INTRODUCTION 


1 .  Effective  management  of  a  workforce  depends,  in  part  at 
least,  on  managers'  having  an  accurate  knowledge  of  the  tasks 
that  their  personnel  are  expected  to  be  capable  of  performing. 

The  RAAF's  main  method  of  collecting  this  information  is  through 
its  ongoing  CODAP  occupational  analysis  (OA)  programme.  The  data 
from  these  OA  programmes  are  used  to  produce  two  series  of 
important  documents;  trade  specifications  and  course  graduation 
requirements.  Both  of  these  documents  enable  management  to 
determine  the  tasks  that  their  members  are  expected  to  be  capable 
of  performing  and  the  minimum  proficiency  that  their  members 

are  required  to  achieve  in  each  task.  The  following  paragraphs 
describe  these  documents. 

2.  Trade  Specifications.  The  RAAF  sets  trade  specifications 

to  ensure  that  designated  tasks  are  performed  at  the  required 
level  of  proficiency.  A  series  of  RAAF  publications  contains  the 
Trade  specifications  for  all  RAAF  musterings  up  to  and  including 
the  rank  of  corporal.  The  trade  specification  encompasses: 

a.  a  general  description  of  the  mustering, 

b.  entry  requirements  and  method  of  entry, 

c.  a  career  plan, 

d.  the  tasks  comprising  the  trade  standard,  and 

e.  the  trade  tests  which  must  be  passed  at  each  level. 

Most  of  the  effort  required  to  produce  the  trade  specification 
for  a  mustering  goes  into  producing  the  trade  standard.  A  trade 
standard  details  those  tasks  that  an  airman  is  expected  to 
perform  at  each  rank  level  from  aircraftman  to  corporal  and  also 
the  associated  performance  level  in  each  task.  The  proficiency 
levels  of  the  tasks  are  minimum  acceptable  levels  for  entry  to  a 
mustering  or  rank/proficiency  level.  The  trade  standard  also 
incorporates  many  of  the  skills  acquired  once  the  member  has 
completed  his  or  her  basic  trade  training  course. 

3.  Course  Graduation  Requirement.  The  course  graduation 
requirement  is  structured  into  two  parts  as  follows: 
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a.  Part  1  Course  Details  and  Special  Requirements, 
and 

b.  Part  2  Course  Terminal  Objectives. 

Part  1  identifies  the  course,  specifies  the  aim  of  the  course, 
states  the  elegibility  conditions  and  graduation  status  of 
trainees,  stipulates  any  constraints  of  a  policy  nature,  and 
specifies  the  appointment  of  a  sponsor.  The  main  concern  of  this 
paper  is  with  Part  2  of  the  graduation  requirement.  Part  2  lists 
the  course  terminal  objectives  which  specify  the  training  goals 
in  student  oriented  terms.  Course  terminal  objectives  are 
statements  of  what  the  trainee  must  be  able  to  do,  and  how  well, 
to  graduate  from  the  course.  Given  this  information,  all 
concerned  with  the  training  system  -  managers,  instructors, 
students,  examiners  and  employers  -  should  know  precisely  what 
a  course  is  designed  to  achieve. 

4<  Having  described  these  two  documents,  I  will  now  discuss 

the  production  and  validation  of  what  I  believe  is  the  most 
important  part  of  each  -  the  trade  standards  and  the  course 
terminal  objectives. 


AIM 


5.  The  aim  of  this  paper  is  to: 

a.  Describe  the  content  and  uses  of  trade  standards 
and  course  terminal  objectives. 

b.  Explain  the  method  devised  for  producing  these 
documents  from  OA  data. 

c.  Explain  why  this  method  is  sometimes  difficult 
to  apply. 

d.  Outline  advantages  of  having  OA  data  available 
during  the  production  of  trade  standards  and  course 
terminal  objectives. 


DEFINITION  OF  TERMS 


6.  As  some  members  of  the  audience  may  not  be  familiar  with 

all  the  RAAF  terms  used  in  this  paper,  I  will  define  these  terms 
before  proceeding  to  cover  the  aims  of  this  paper. 


120  - 


a.  Trade.  A  trade  is  a  group  of  related  jobs. 

It  is  a  classification  into  which  an  airman 
is  placed  comensurate  with  previous  civilian 
employment  or  RAAF  training.  The  term  is  used 
synonymously  with  the  term  mustering. 

b.  Job.  A  job  refers  to  what  an  individual  airman 
does  as  part  of  his  usual  employment  in  the  RAAF 
workforce.  A  job  is  composed  of  a  number  of 
related  duties. 

c.  Duty.  A  duty  is  a  large  segment  of  a  job,  and  is 
composed  of  a  number  of  discrete  related  tasks. 

d.  Task.  A  task  is  a  discrete  action  or  specific 
unit  of  work. 

e.  Trade  Test.  A  trade  test  is  a  means  by  which  the 
performance  of  an  airman  is  evaluated  for  either 
entry  to  a  trade  or  for  advancement.  The  trade 
test  is  designed  to  assess  the  capability  of  an 
airman  to  perform  tasks  covered  in  the  applicable 
trade  standard.  Trade  tests  usually  comprise 

a  practical  and  a  written  test. 

f.  Career  Flan.  Annex  A  illustrates  the  general 
career  plan  for  RAAF  airmen. 


TRADE  STANDARDS  AND  COURSE  TERMINAL  OBJECTIVES 


The  Trade  Standard 

7.  What  the  Trade  Standard  should  Include.  As  laid  down  in 
the  applicable  RAAF  publication,  the  trade  standard  is  to  include: 

a.  All  those  tasks,  and  only  those  tasks,  an  airman 
is  expected  to  perform  at  each  rank  level. 

b.  Those  subjects  and  skills  the  knowledge  or 
application  of  which  is  necessary  for  performance 
of  mustering  tasks  but  which  are  not  apparent 
from  task  statements. 

8.  As  an  example  to  illustrate  the  content  of  sub-paragraph 
7b,  suppose  a  task  for  one  of  the  radio  trades  is  "Diagnose 
faults  in  radio  receivers"  Here  a  knowledge  of  the  subject 
'electronics'  is  self  evident  and  would  not  be  stated  in  the 
trade  standard.  However,  whilst  doing  the  task  the  airman  must 
apply  the  relevant  safety  precautions;  these  can  be  made  more 
explicit  by  listing. 
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9.  What  the  Trade  Standard  Should  Exclude.  With  the  exception 

of  Technical  Administration,  Safety,  and  Aircraft  Ground  Handling 
(tasks  common  to  all  technical  trades),  the  Trade  Standard  will 
contain  information  relating  to  one  mustering  only.  Generally,  it 
will  exclude: 


a.  the  subjects  of  promotion  examinations, 

b.  skills  and  knowledge  gained  through  special 
courses  limited  to  selected  airmen, 

c.  the  peculiar  requirement  of  individual  locations, 

d.  variations  of  skill  and  knowledge  arising  from 
the  peculiarities  of  particular  equipment  types, 
and 


e.  contingency  tasks.  (1) 

(1)  The  trade  standard  should  not,  for  example, 
allocate  a  sergeant's  tasks  to  a  corporal, 
in  case  there  should  not  be  a  sergeant 
available.  The  aim  of  the  trade  standard 
is  to  establish  minimum  acceptable  levels  of 
skills.  It  is  not  intended  to  provide  for 
every  conceivable  eventuality. 

10.  Uses.  The  trade  standard  informs  section  commanders  of 

the  performance  they  can  reasonably  expect  from  new  or  newly 
promoted  members.  These  managers  are  then  in  a  position  to  assess 
fairly  the  member's  work,  and  so  ensure  he  gains  additional  on-the- 
job  training  and  experience  to  enhance  his  or  her  effectiveness. 

The  trade  standard  is  also  available  to  the  member  to  indicate  the 
range  of  tasks  he  is  expected  to  carry  out,  and  the  performance 
expected  on  each  task.  This  information  is  of  assistance  to 
the  member  when  he  or  she  is  preparing  for  a  trade  test. 


Course  Terminal  Objectives 

1 1 .  What  Should  be  Included.  A  prime  requirement  of  course 

terminal  objectives  is  that  they  accurately  reflect  the  skills  and 
knowledge  required  for  the  task  on  which  members  of  the  trade 
are  likely  to  be  employed  in  the  first  two  or  three  years 
after  graduation.  Thus,  the  course  terminal  objectives  must  be 
based  on  a  thorough  knowledge  and  understanding  of  the  jobs  for 
which  the  trainee  is  being  prepared.  As  these  jobs  may  change 
before  the  trainee  graduates,  an  allowance  must  be  made  for 
forthcoming  technology  associated  with  new  aircraft  or  equipment 
which  will  be  entering  the  service.  In  addition,  skills  the 
graduate  would  require  in  an  emergency  or  wartime  situation 
should  be  considered  for  inclusion  in  the  course  terminal 
objectives. 


12.  What  Should  be  Excluded.  As  mentioned  previously,  the 

emphasis  in  a  course  terminal  objective  is  on  what  a  trainee  must 
De  able  to  do  at  the  end  of  his  training  course.  Objectives  or 
practical  skills  that  are  essential  to  the  achievement  of  a 
terminal  objective  are  not  included  in  the  course  terminal 
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objectives,  as  these  would  be  obtained  through  task  analysis 
during  the  syllabus  construction.  For  example,  if  'Select  and 
operate  appropriate  fire  fighting  equipment  in  a  simulated 
situation'  is  a  course  terminal  objective,  then  objectives  such 
as  'State  the  different  classes  of  fire  and  the  appropriate  method 
of  extinguishing  them'  would  not  be  included  with  the  course 
terminal  objectives  as  this  knowledge  is  essential  to  the  performance 
of  the  stated  terminal  objective.  However,  this  cognitive  skill 
is  an  enabling  objective  and  would  be  listed  in  the  syllabus  of 
training. 

13.  Uses.  The  training  school  composes  the  syllabus  of 

training  for  the  course  from  the  course  terminal  objectives 
contained  in  the  graduation  requirement  for  the  mustering.  As 
course  terminal  objectives  state  only  the  end  product  of  training, 
they  need  to  be  scrutinized  to  identify  the  enabling  skills  and 
knowledge  needed  to  be  acquired  by  the  trainee  before  he  performs 
task  at  the  required  proficiency  level.  The  process  of  identifying 
these  enabling  objectives  is  commonly  called  task  analysis.  The 
syllabus  of  training  can  be  used  by  all  involved  with  training 
(managers,  instructors,  students  and  examiners)  to  determine  the 
detailed  course  content. 


PROCEDURE  DEVISED  FOR  USING  OA  DATA  TO 
PRODUCE  TRADE  STANDARDS  AM)  COURSE  TERMINAL  OBJECTIVES 


14.  Before  outlining  the  procedure  developed  by  the  RAAF  OA 
Cell  to  assist  in  the  production  of  trade  standards  and  course 
terminal  objectives,  I  will  briefly  outline  the  methods  the  RAAF 
uses  to  collect  its  OA  data  and  the  format  of  the  data  used  to 
produce  these  documents.  This  outline  will  assist  in  the  ex¬ 
planation  of  the  methodology  developed. 

OA  Data  Collection 

15.  In  the  RAAF,  the  OA  survey  booklets  are  normally  admini¬ 
stered  to  over  80  percent  of  the  mustering  being  surveyed.  Each 
members  selected  to  complete  the  survey  booklet  fills  out  a 
background  information  section,  checks  the  task  he  performs  and 
indicates  how  his  worktime  is  distributed  across  the  tasks  in 
his  job.  In  addition,  about  45  senior  non-commissioned  officers 
(Flight  Sergeants  and  Sergeants)  rate  tasks  in  the  inventory  on 
one  or  more  of  the  following  factors:  task  learning  difficulty 
(TLD),  consequence  of  inadequate  performance  (CIP),  and  need  for 
immediate  performance  (NIP).  The  data  after  collection  are  then 
analysed  using  the  suite  of  CODAP  programmes  into  the  required 
outputs. 

Procedure  -  Trade  Standards 


16.  Now,  let's  look  at  one  of  these  outputs  with  the  purpose 

of  producing  these  two  documents.  The  printout  used  lists  the 
tasks  performed  by  ACs,  LACs,  and  CPLs  ranked  in  order  of  the 
percentage  of  ACs  who  perform  them.  The  printout  also  lists  any 
ratings  on  task  factors  that  have  been  collected.  Table  1  presents 
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an  extract  from  such  a  printout  for  the  Photographer  Trade. 


TABLE  1 


TYPICAL  PRINTOUT  USED  FOR 
PRODUCTION  OF  TRADE  STANDARDS 
AND  COURSE  TERMINAL  OBJECTIVES 


TASKS 

PERCENT 

PERTX3RMING 

TLD 

NIP 

(FIRST  PAGE) 

AC 

LAC 

CPL 

Clean  equipment  after 
chemical  mixing 

90 

79 

70 

2.8 

6.3 

Dry  or  glaze  projection  or 
contact  prints 

70 

63 

65 

3.3 

5.7 

Prepare  B/W  film  for  processing 
in  dark  room 

60 

68 

70 

4.4 

6.1 

Plot  characteristic  curve 
using  densitometer 

55 

58 

34 

6.5 

7.2 

(MIDDLE  PAGE) 

Process  B/W  prints  by  machine 

35 

16 

30 

4.7 

5.4 

Expose  air  film  negatives  on 
10"x10"  projection  printer 

25 

26 

4 

6.2 

5.1 

Copy  photographs  using 
process  camera 

20 

32 

39 

6.1 

5.3 

(LAST  PAGE) 


Install  or  remove  type  B  air 
came ra 

0 

5 

9 

6.3 

4.6 

Perform  equipment  schedule 
checks 

0 

0 

26 

5.3 

5.4 

Process  colour  transparencies 
manually 

0 

5 

13 

6.6 

6.4 

Service  type  A  magazine 

0 

5 

13 

5.7 

5.7 

17.  The  first  column  of  values  gives  the  percentage  of  ACs 

performing  each  task  as  part  of  their  normal  job.  It  also  gives 
the  probability  that  an  AC  Photographer  will  be  required  to 
perform  each  of  the  tasks  listed.  You  will  observe  that  tasks  on 
the  first  page  of  the  printout  are  all  encountered  by  at  least 
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55  percent  of  AC  Photographers.  As  there  is  a  high  probability 
that  an  AC  photographer  will  encounter  each  of  these  tasks,  one 
would  expect  to  find  most  of  these  tasks  contained  in  the  AC 
trade  standard  for  Photographers  and  also  in  the  terminal 
objectives  for  the  Basic  Photographer’s  Course.  In  contrast,  the 
last  page  of  the  printout  lists  tasks  encountered  by  relatively 
few  ACs  and,  as  most  ACa  are  unlikely  to  encounter  these  tasks, 
one  would  expect  that  few  of  these  tasks  would  be  included  in  the 
trade  standard  (or  in  course  terminal  objectives).  In  fact, 
many  of  the  tasks  on  the  last  page  should  be  excluded  from  the 
trade  standard  purely  on  the  grounds  that  they  reflect  peculiar 
requirements  of  individual  locations.  (Certainly,  it  would  not 
be  cost  effective  to  train  all  students  in  these  tasks  as  fewer 
than  five  percent  of  graduates  will  encounter  them  in  their  first 
posting  after  graduation. ) 

18.  The  procedure  that  has  been  recommended  for  adoption  in 
the  production  of  trade  standard  is  as  follows.  Firstly,  a  trade 
standards  review  team,  composed  of  at  least  five  SNCOs  or  WOFFs, 
under  the  guidance  of  an  officer  from  Trade  Standards  and  Trade 
Testing  Section,  uses  the  data  together  with  policy  and  trade 
specialists  inputs  to  determine  tasks  that  should  be  included 

in  the  draft  trade  standard.  This  is  achieved  by  the  review  team 
using  the  percentage  of  ACs  performing  the  task  as  a  guide  to  the 
probability  that  an  AC  will  be  required  to  perform  the  task.  The 
team  then  considers  whether  the  task  should  be  included  in  the 
draft  trade  standard  at  AC  level  or  not.  If  the  concensus  of 
opinion  from  the  members  of  the  review  team  is  that  all  ACs  should 
be  capable  of  performing  the  task,  then  it  would  be  included  in 
the  draft  trade  standard  for  ACs.  Otherwise,  the  task  would  not 
be  included  in  the  draft.  The  task  is  then  considered  in  turn 
for  inclusion  in  the  draft  trade  standards  for  LACs  and  CPLs. 

19.  Next  the  review  team  determines  the  proficiency  levels 
required  for  each  task  for  the  three  ranks.  The  task  factors 
(task  learning  difficulty,  consequences  of  inadequate  performance, 
and  need  for  immediate  performance)  together  with  the  trade 
experience  of  the  members  of  the  review  team  are  used  to  determine 
these  proficiency  levels. 

20.  The  resulting  draft  trade  standard  is  subsequently  examined 
by  the  specialist  staff  officers  for  the  trade  to  ensure  that  it 
not  only  meets  the  existing  requirements  but  also  incorporates  any 
task  or  trade  structure  changes  likely  to  affect  the  mustering 
before  the  next  trade  standards  review. 

Procedure  -  Course  Terminal  Objectives 

21 .  Before  outlining  the  procedure  recommended  for  producing 
course  terminal  objectives,  I  will  explain  the  relationship  between 
the  two  documents.  There  should  be  a  close  relationship  between 
trade  standards  (which  list  the  tasks  members  are  expected  to 
perform)  and  the  course  terminal  objectives  (which  indicate  the 
training  given  to  prepare  members  for  performing  tasks).  This 


I 


relationship  should  be  especially  close  between  the  AC  trade 
standard  and  the  terminal  objectives  for  the  basic  trade 
training  course.  (In  these  circumstances,  when  the  member 
graduates  off  course  as  an  AC,  he  should  have  attained  the  AC 
trade  standard.) 

22.  The  procedure  recommended  for  producing  course  terminal 

objectives  directly  related  to  a  trade  standard  is  for  a  team 
(preferably  consisting  of  some  members  who  helped  produce  the 
trade  standard)  to  produce  the  course  terminal  objectives  from 
the  appropriate  trade  standard.  They  will  determine  if  there 
are : 


a.  any  tasks  in  the  trade  standard  for  which  formal 
training  is  not  needed  (because  the  required 
proficiency  is  a  pre-requisite  for  course 
selection,  the  tasks  are  better  learned  on-the- 
job,  or  for  which  formal  training  would  not  be 
cost-effective) ; 

b.  any  tasks  on  the  LAC  or  CPL  trade  standard  for 
which  it  may  be  more  cost-effective  to  give 
training  on  the  basic  course.  (Thus  some 
overtraining  may  be  justified  if  it  eliminates 
the  need  for  future  courses); 

c.  any  additional  training  topics  required  as  a 
result  of  new  technology  equipment  about  to 
be  introduced  into  the  RAAF;  and 

d.  any  other  tasks  that  should  be  introduced 
according  to  the  current  policy  regarding  the 
mustering. 

Problems  Encountered  Using  Devised  Procedures 

23.  Although  the  application  of  the  procedures  for  using  the 

OA  data  to  produce  trade  standards  and  course  terminal  objectives 
greatly  simplifies  the  task  of  producing  both  documents,  experience 
has  shown  that  the  procedure  is  not  as  simple  to  apply  in  practice 
as  had  been  hoped.  The  task  of  producing  the  documents  is  made 
very  difficult  by  the  high  heterogeneity  found  in  most  RAAF 
musterings  (particularly  the  technical  musterings).  For  example, 
in  the  OA  survey  on  the  Instrument  Fitter  mustering  Table  2  we 
found  that  for  corporals  all  tasks  of  a  technical  nature  were 
being  performed  by  fewer  than  44  percent  of  the  mustering.  With 
such  heterogeneous  musterings,  the  process  of  deciding  which 
tasks  should  be  included  in  the  trade  standard  (or  for  that  matter 
the  course  terminal  objectives)  is  more  difficult  than  with  a 
homogeneous  mustering  (such  as  the  Photographer  mustering  OA 
survey  data  for  Corporals  shown  in  Table  3).  In  addition,  the 
trade  review  panels,  when  faced  with  these  difficult  decisions, 
prefer  merely  to  update  the  old  trade  standard  from  their  experience 
rather  than  attempt  to  us-  the  available  OA  data. 
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TABLE  2 

EXAMPLE  OF  HETEROGENEOUS  MUSTERING;  TOP  30  TASKS  IN 
TERMS  Qfr  PtiRCEtiTAGE  OF  "INSTRUMENT  FITTER  CORPORALS  PERFORMING 


TASKS  PERCENT 

PERFORMING 

Participate  on  parades  84% 

Perform  base  rostered  duties  other  than  duty  crew  73% 

Clean  section  73% 

Allocate  personnel  to  duties  or  tasks  64% 

Raise  demands  for  equipment  56% 

Give  on  the  job  training  56% 

Amend  publications  or  orders  53% 

Secure  buildings  49% 

Search  equipment  catalogue  for  item  identification  48% 

number 

*  Inspect  pitot  static  systems  43% 

*  Inspect  pitot  heads  42% 

*  Carry  out  after  flight  checks  42% 

*  Prepare  aircraft  for  flight  41% 

*  Service  pitot  static  system  41% 

*  Inspect  pitot  static  plumbing  41% 

Control  security  of  section  buildings  40% 

Complete  appropriate  section  of  maintenance  documents  39% 

*  Fire  SLR  at  a  fixed  target  on  an  open  range  39% 

*  Troubleshoot  pitot  static  plumbing  38% 

*  Inspect  oxygen  regulators  38% 

*  Service  tacho  indicator  38% 

Insert  line  amendments  into  service  publications  37% 

Move  or  park  aircraft  37% 

Marshall  aircraft  36% 

*  Remove  or  install  tacho  indicators  36% 

*  Refuel  or  defuel  aircraft  35% 

*  Inspect  instrument  panels  35% 

*  Inspect  thermocouple  temperature  system  35% 

*  Service  clocks  35% 

*  Troubleshoot  gyromagnetic  compass  system  34% 


*  Note:  Only  these  tasks  (16  out  of  30)  are  of  a  technical 

nature.  This  means  that  all  other  technical  tasks 
in  the  Instrument  Fitter  mustering  are  performed  by 
fewer  than  35%  of  Corporals. 
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TABLE  3 

EXAMPLE  OP  HOMOGENEOUS  MUSTERINGS:  TOP  30  TASKS  IN  TERMS 
OF  PERCENTAGE  OF  PHOTOGRAPHER  CORPORALS 
PERFORMING 


f 


t 


TASKS  PERCENT 

PERFORMING 

Participate  on  parades  96% 

Carry  out  cleaning  of  working  area  91% 

Maintain  security  of  section  buildings  78% 

*  Trim  projection  on  contact  points  78% 

*  Maintain  photographic  job  register  74% 

Participate  in  production  conference  within  section  74% 
or  unit 

*  Prepare  B/W  chemistry  for  printing  in  dark  room  70% 

*  Prepare  B/W  film  for  processing  in  dark  room  70% 

*  Clean  equipment  after  chemical  mixing  70% 

*  Process  B/W  film  manually  70% 

*  Clean  negatives  before  or  during  printing  70% 

*  Prepare  or  maintain  photographic  records  70% 

Carry  out  rostered  duties  70% 

*  Unload  hand  held  camera  70% 

*  Select  safelight  filter  65% 

*  Expose  print  ground  negatives  on  contact  printer  65% 

*  Wash  projection  or  contact  prints  using  tray  set-up  65% 
or  rotary  print  washer 

*  Dry  or  glaze  projection  or  contact  prints  65% 

*  Sort  projection  or  contact  prints  65% 

Raise  photographic  work  order  65% 

Maintain  section  workload  or  production  tables  65% 

*  Stamp  and  title  photographic  prints  65% 

*  Maintain  negative  register  65% 

*  Act  as  photographic  adviser  65% 

Counsel  subordinates  on  job-related  problems  65% 

*  Monitor  usage  of  photographic  materials  61% 

*  Store  working  chemicals  61% 

*  Prepare  for,  set  up  and  take  group  photographs  61% 

excluding  PR  photographs 

*  Prepare  for,  set  up  and  take  identification  61% 

photographs 

*  Prepare  for,  set  up  and  take  PR  photographs  61% 


*  Note;  22  out  of  30  tasks  are  technical.  After  30  tasks 

there  are  still  60%  of  corporals  performing  tasks. 
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24.  The  RAAF  is  a  very  small  force  (22,500)  compared  with 
the  air  forces  of  other  nations,  and  yet  the  RAAF  has  to  provide 
engineering  and  maintenance  support  for  a  wide  range  of 
operational  roles.  In  turn,  these  roles  are  spread  over  a  wide 
range  of  aircraft  types,  weapons  systems,  and  supporting  systems. 
Within  this  environment  tradesmen  are  considered  to  be  fully 
negotiable,  ie,  all  corporal  Instrument  Fitters  can  be  posted 

to  any  type  of  aircraft  or  other  system  which  is  established 
for  the  trade  and  rank  for  work  at  any  depth  of  maintenance. 

This  accounts  for  the  high  heterogeniety  in  many  RAAF  musterings. 

25.  The  flexibility  the  RAAF  enjoys  in  posting  members  is 
made  possible  as  a  result  of  the  high  training  investment  it 
carries;  any  drastic  reduction  in  the  range  of  taks  in  the 
trade  standard  or  course  terminal  objectives  would  reduce  this 
posting  flexibility.  If  this  flexibility  is  required,  then  it 
must  be  paid  for  in  terms  of  extra  training  costs.  Cost-effective 
training  can  probably  only  be  achieved  by  accepting  the  concept 

of  training  members  specifically  for  the  job  they  will  do  on  their 
first  appointment.  The  courses  resulting  from  this  approach 
would  be  shorter  and  more  practically  oriented  and  would  give 
the  RAAF  more  productive  service  (for  every  training  dollar  spent) 
from  its  first  tour  (and  also  its  only  tour)  airmen.  However, 
the  concept  of  cost-effective  training  is  in  conflict  with  that  of 
posting  flexibility.  What  we  in  the  RAAF  should  be  trying  to  do 
is  balance  the  need  for  flexibility  against  the  cost  of  overtraining 
when  deciding  the  contents  of  trade  standards  (and  hence  trade 
training  courses). 

26.  Another  one  of  the  problems  that  has  arisen  has  been  caused 
by  the  course  terminal  objectives'  being  produced  directly  from 

OA  data  concerning  the  trade  without  reference  to  the  relevant 
trade  standard.  As  the  same  form  of  printout  is  used  to  produce 
the  course  terminal  objectives  as  the  trade  standard,  and  the 
general  procedures  adopted  for  selecting  tasks  for  inclusion  in 
each  document  are  similar,  one  would  expect  to  find  a  readily 
discernable  common  content  in  these  documents.  However,  this 
relationship  is  difficult  to  see. 

27.  The  reasons  why  the  required  relationship  cannot  be  readily 
seen  is  attributable  to  the  fact  that  different  teams  work 
independently.  Although  each  team  uses  the  same  OA  data  to  help 
produce  each  document,  the  concensus  decisions  made  on  whether 

to  include  a  task  in  draft  documents  or  not  is  based  on  the 
opinions  of  members.  In  addition  each  teams  tends  to  group  several 
tasks  in  the  draft  document  together  under  the  one  general  heading 
and  the  classification  used  depends  on  the  team. 

28.  We  in  the  RAAF  OA  Section  believe  that  if  the  required 
relationship  between  trade  standards  and  course  terminal  objectives 
is  not  only  to  exist  but  also  readily  be  seen  to  exist,  then  it  is 
essential  that  the  production  of  these  documents  be  closely 
co-ordinated.  Unless  their  production  is  closely  co-ordinated, 
the  documents  could  easily  end  up  being  incompatible. 
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29.  Unfortunately,  in  the  RAAF  there  are  two  schools  of  thought 
as  to  which  documents  should  be  produced  first  and  considerable 
discussion  has  taken  place  on  this  issue  without  its  being 
resolved.  Further,  the  additional  effort  required  to  solve  this 
issue  would  only  result  in  the  production  of  these  documents 

being  delayed  and  is,  therefore,  not  warranted  particularly  as 
most  of  the  objections  from  both  sides  can  be  overcome  and  the 
required  relationship  between  the  documents  obtained  if  both 
documents  are  produced  concurrently  by  the  same  team.  (This 
suggestion  would  also  save  some  duplication  of  effort  by  both 
teams  and,  hence,  would  also  result  in  financial  savings). 

Benefits  of  Using  OA  Data 

30.  Despite  the  problems  associated  with  using  OA  data  as 
outlined  above,  there  have  been,  and  will  continue  to  be,  positive 
contributions  from  OA  to  trade  standards  and  training.  OA  data 
have  enabled  those  involved  in  producing  these  documents  to  make 
decisions  based  on  what  the  members  of  a  musterings  were  actually 
doing  at  the  time  of  the  survey.  In  the  past  these  decisions  had 
to  be  made  in  the  absence  of  such  reliable  information  and,  as 
far  as  training  courses  were  concerned,  the  decisions  erred  on  the 
wide  of  conservatism.  (That  is,  when  course  content  was  determined, 
extra  training  was  often  included  just  to  be  on  the  safe  side). 
Decisions  made  under  these  circumstances  have  resulted  in  some 
unnecessary  and  misdirected  training  being  included  in  the  course 
content . 

31 .  Providing  CO DAP  OA  data  that  tells  us  precisely  what  workers 
were  doing  on  the  job  at  the  time  of  the  survey,  has  enabled  our 
training  and  trade  testing  system  to  be  more  job  related,  helped 

to  identify  areas  of  overtraining  (where  the  training  was  not  nor 
was  likely  to  be  used),  and  enabled  policy  makers  to  make 
decisions  concerning  the  responsibility  of  the  mustering. 

32.  Finally,  if  the  RAAF  decides  to  continue  with  overtraining 
in  some  areas  to  maintain  posting  flexibility,  then  it  is  important 
that  those  involved  in  determining  course  content  have  some  measure 
of  how  much  extra  flexibility  will  be  obtained  for  the  additional 
cost  involved.  OA  data  on  the  mustering  can  help  by  indicating  how 
much  extra  training  will  be  required  to  provide  the  required  degree 
of  posting  flexibility. 


CONCLUSION 


33*  This  paper  has  outlined  the  method  used  by  the  RAAF  to 

produce  trade  standards  and  course  terminal  objectives  and 
explained  the  difficulties  that  have  been  experienced  in  applying 
the  method  to  many  of  the  RAAF  trades.  One  of  these  difficulties 
is  the  requirement  to  balance  the  need  for  flexibility  in  posting 
members  a-'ainst  the  cost  of  overtraining  the  members.  In  spite 
of  these  difficulties,  CODAP  OA  data  has  been  positive  contri¬ 
butions  to  trade  standards  and  training.  By  providing  accurate 
data  on  which  decisions  can  be  made,  OA  has  helped  remove  much  of 
the  'guess  work'  out  of  the  copilation  of  trade  standard  and 
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course  terminal  objectives.  This  information  will  provide  manage¬ 
ment  with  an  accurate  knowledge  of  the  tasks  their  personnel  are 
expected  to  be  capable  of  performing. 

34.  When  the  RAAP  acquired  the  complete  CODAP  package  in 

1974,  the  full  potential  of  OA  was  realized,  subsequently  some 
progress  has  been  made  in  developing  methods  of  using  OA  data 
in  the  RAAF.  The  task  that  lies  ahead  is  to  convince  others  of 
the  benefit  to  be  gained  from  using  CODAP  data. 
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PROPOSED  METHOD  OF  JOB  EVALUATION 
FOR  PAY  PURPOSES 
FOR  "OTHER  RANKS"  JOBS 


Wing  Commander  J.W.K.  Fugill  (Ret)  B.A.  M . Ed 


1  -  The  evaluation  of  jobs  for  pay  purposes  is  a  conceptually 

difficult  and  controversial  area  of  management.  Anyone  proposing 
an  entirely  new  approach  to  job  evaluation  not  only  sets  himself 
an  almost  unattainable  goal,  but  tends  to  alienate  even  those 
people  whose  interests  he  is  supposed  to  represent.  Hot  the  least 
distasteful  aspect  of  this  procedure  is  the  implication  that  those 
who  support  the  existing  system  are  conceptually  wrong  -  headed 
and  somewhat  less  perspicacious  than  they  ought  to  be.  In  redress, 
this  paper  is  dedicated  to  those  position  description  writers  and 
evaluators  who  did  their  best  with  the  present  system  during  the 
long  night  of  1974,  1975  and  1976. 

2.  Following  the  publication  of  the  Job  Evaluation  Report 

for  Other  Ranks  in  July  1976,  a  review  of  the  existing  system  was 
undertaken.  When  it  became  clear  that  piece  -  meal  modifications 
would  not  satisfy  the  Services'  needs,  research  was  begun  on  a 
replacement  system,  henceforth  known  as  the  Australian  Defence  Force 
System. 


AIMS 


3.  The  aims  of  this  paper  are  to: 

a.  explain  the  rationale  for  a  job  evaluation 
system  that  is  specific  to  the  military 
industry,  and 

b.  describe  the  conceptual  framework  of  the 
proposed  system. 


RATIONALE  FOR  A  MILITARY 
JOB  -EVALUATION  SYSTEM- 


Future  Need  for  Job  Evaluation 


4.  In  the  future,  the  Services  could  create  new  employment 

categories  or  restructure  existing  categories  in  response  to  changes 
in  technology  and  operational  procedures.  Those  actions  would  create 
a  need  for  supplementary  reviews  of  pay  and  allowances.  To  ensure 
consistent  application  of  the  Kerr/Woodward  principles,  the  job 
evaluation  system  will  have  to  fit  the  new  categories  onto  an  exist¬ 
ing  benchmark  scale. 
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5.  In  addition,  there  is  the  possibility  that  a  comprehensive 

job  evaluation  review  could  be  initiated  at  any  time.  In  that  event, 
the  Services  should  have  researched  and  defensible  system  with  which, 
to  properly  represent  the  interests  of  their  members.  In  this  sense, 
this  proposal  is  a  contingency  plan  capable  of  implementation  with 
the  minimum  of  inconvenience. 


Traditional  Evaluation  Systems 

6.  A  review  of  traditional  evaluation  systems  (including  the 

twelve  proposals  submitted  in  competition  with  Hay  Associates) 
revealed  a  number  of  major  deficiencies. 

a.  The  sample  of  jobs  that  can  be  surveyed  economically 
represent  only  a  small  percent  of  each  category. 

In  a  category  comprising  numerous  job-types,  it  is 
very  difficult  to  decide  which  position  description 
is  a  "typical"  representation  of  a  category  as  a 
whole. 

b.  Evaluation  factors  have  a  measure  of  face  validity, 
but  little  statistical  support  as  to  reliability. 
Thus,  although  Know-How,  Problem  Solving  and  Account¬ 
ability  have  a  superficial  attractiveness,  how  do 

we  know  that  they  are  appropriate? 

c.  Where  factors  are  weighted,-  the  derivation  of  the 

weights  is  not  explained.  The  Hay  Associates  contact 

did  not  state  what  weight  were  being  applied  for  the 
three  factors  Know-How,  Problem-Solving  and  Account¬ 
ability.  After  the  1976  evaluation  exercise,  an 
attempt  was  made  to  infer  how  the  factor  scores  had 
been  distributed.  The  general  pattern  that  emerged 
was  as  follows: 

Know-How  -  10%> 

Problem-Solving  -  1  2  %c 

Accountability  -  1 8% 

The  over-riding  importance  given  to  Know-How  in 
determining  the  final  score  is  obvious.  Note¬ 
worthy  also  is  the  correlation  co-efficient  of  0.95 
between  the  Know-How  scores  and  the  Total  Scores  for 
each  category.  Such  a  high  overlap  suggests  that 
essentially  the  same  benchmarks  scale  could  have  been 
derived  without  the  cost  of  evaluating  and  scoring 
the  Problem-Solving  and  Accountability  factors. 

Stated  another  way,  the  benchmark  scale  which  con¬ 
tributed  to  the  work  value  study  was  basically  a 
Know-How  scale. 
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d.  The  scoring  is  usually  done  by  hand.  When  large 
representative  samples  are  involved,  this  becomes  a 
laborious  and  time-consuming  task. 

e.  No  provision  is  made  for  report  the  extend  of  agree¬ 
ment  between  the  scorers.  Therefore,  managers  do  not 
know  what  degree  of  confidence  can  be  placed  on  the 
scorers'  decisions.  In  the  case  of  the  Hay  scoring 
exercise,  the  three  scorers  may  well  have  been  highly 
idiosyncratic  in  their  views  on  the  nature  of 
military  work. 

f.  The  systems  are  designed  primarily  for  the  civilian 
workforce. 

g.  Because  the  systems  are  manpower  intensive,  the  cost 
of  broadly  representative  surveys  tend  to  be  pro¬ 
hibitive.  For  example,  the  preparation  of  a  single 
position  description  will  cost  between  $50  and. $100. 
This  is  an  expensive  investment,  especially  as  the 
position  descriptions  have  little  value  for  other 
management  applications. 

Definition  of  the  Benchmark  Scale 


7.  The  proposed  job  evaluation  system  will  measure  the  relative 
worth  of  all  employment  categories  within  the  Australian  Defence 
Force.  Relative  worth  refers  to  the  contribution  made  by  a  cate¬ 
gory  to  the  achievement  of  Service  aims.  The  arrangement  of  the 
categories  on  a  scale  from  lowest  relative  worth  to  highest  relative 
worth  will  constitute  the  benchmark  scale.  This  scale  will  be  one 
source  of  guidance  in  pay  determinations. 

Evaluation  System  Criteria 

8.  From  the  review  of  traditional  systems  and  from  comments 
made  by  Service  Offices,  the  following  points  emerge  as  likely 
criteria  to  be  met  in  the  Australian  Defence  Force  System. 

9.  Sample  Size.  Category  samples  size  must  be  sufficiently 
large  to  cover  all  significant  job-types.  The  calculated  mean 
score  for  the  sample  is  then  a  fair  representation  of  a  category's 
relative  worth. 

10.  Evaluation  Factors.  Evaluation  factors  must  have  face 
validity  for  "military  skills"  categories  as  well  as  for  categories 
closely  related  to  civilian  trades.  In  addition,  the  factors  must 
be  such  that  reliable  data  can  be  gathered  economically. 

1 1 .  Factor  Weights.  Applied  factor  weights  must  be  derived 
systematically  from  the  broadest  possible  concensus  of  opinion. 

12.  Scoring  Methods.  Scoring  methods  must  be  systematic, 
computerized,  and  subject  to  verification. 
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13.  Judgement  Analysis.  When  judgements  are  involved,  the 
levels  of  agreement  must  be  reportable. 

14.  Costs.  Costs  must  not  be  prohibitive;  this  implies 
minimum  use  of  manpower  and  maximum  use  of  support  systems. 

15.  Military  Orientation.  The  Services  must  be  satisfied 
that  the  system  measures  the  unique  contributions  made  by  sailors, 
soldiers  and  airmen  in  achieving  the  Services'  aims. 

16.  In  addition,  it  is  desirable  that  the  data  be  suitable  for 
other  management  applications. 


THE  PROPOSED  SYSTEM 


17.  This  proposal  is  based  on  a  pilot  study  done  in  1978  with 
the  co-operation  of  the  three  Service  Offices.  However,  the  methods 
used  are  not  new.  The  main  ingredients  are  job  descriptions  from 
the  data  bank,  a  package  of  programs  (CODAP)  and  a  policy-capturing 
model  designed  by  Christal  in  1966. 

18.  The  following  paragraphs  are  a  simplified  version  of  the 
pilot  study  report.  The  basic  steps  in  the  proposed  system  are  as 
follows: 

a.  Select  job  evaluation  factors. 

b.  Score  job  descriptions  which  represent  each 
category. 

c.  Apply  the  agreed  weights  for  each  factor. 

d.  Calculate  the  final  score  for  each  factor. 

e.  Place  the  categories  on  a  benchmark  scale. 


Selection  of  Job  Evaluation  Factors 


19.  For  the  choice  of  factors,  there  are  two  basic  criteria. 

a.  The  factors  must  be  conceptually  acceptable  to 
the  Armed  Forces  Industry;  and 

b.  The  factors  must  be  so  defined  that  consistently 
reliable  information  can  be  gathered  at  the  task 
level. 

20.  So  that  the  pilot  study  could  be  completed  before  this 
seminar,  several  factors  were  chosen  without  complete  agreement 
from  the  three  Services.  Complete  agreement  is  a  possibility,  but 
hardly  likely I 
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21 .  The  first  chosen  factors  were  Task  Difficulty  and 
Consequences  of  Inadequate  Performance.  Task  difficulty  appears 

to  have  considerable  overlap  with  Know-How  in  the  Hay  System.  Both 
factors  require  recognition  of  how  many  bits  of  knowledge,  and  the 
depth  of  knowledge  required  fir  satisfactory  performance.  Con¬ 
sequences  of  Inadequate  Performances  is  conceptually  related  to 
Problem  Solving  and  Accountability.  But  more  importantly,  previous 
experience  in  the  USA  and  Australian  services  shows  that  work  super¬ 
visors  give  highly  reliable  information  on  these  factors  at  the 
task  level. 

22.  The  second  pair  of  factors  were  Hazard  and  Environment. 
These  were  a  unanimous  choice  of  the  Services.  Fortuitously,  in 
1977  the  RAN  had  tested  these  factors  at  shore  installations  and  on 
board  ships.  The  reliability  of  the  judgements  was  found  to  be 
almost  as  high  as  for  Task  Difficulty  and  for  Consequences  of 
Inadequate  Performance. 


Scoring  Category  Job  Descriptions 

23.  Table  I  shows  the  standard  method  of  scoring  Task  Difficulty 

on  a  hypothetical  job  description  for  a  group  of  Girl  Fridays. 


TABLE  I  -  SCORING  TASK  -  DIFFICULTY 


Task 

Percent  Time 
Spent  ( 1  ) 

Relative  Task 
Difficulty (2) 

Cross 

Products (3) 

Answer  telephone 
calls 

5 

1  .2 

6 

Type  letters 

70 

4.5 

315 

Prepare  refreshments 

15 

2.0 

30 

Maintain  file 
register 

10 

3.1 

31 

Stun  of  Cross  Products  =  382 

The  Percent  Time  Spent  values  are  the  averages  for  this  group  of 
workers  (Column  1).  Relative  Task  Difficulty  values  are  provided 
by  work  supervisors  (Column  2).  The  Cross  Products  (Column  3)  are 
added  to  give  an  Average  Task  Difficulty  score. 


I 
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24.  To  complete  this  hypothetical  case,  the  scores  for  the  four 

factors  are  shown  in  TABLE  II. 

TABLE  II  -  FACTOR  SCORES 


Task  Difficulty 

382 

Consequences  of  Inadequate  Performance 

250 

Hazard 

150 

Environment 

125 

In  most  traditional  system,  the  next  step  is  to  add  up  the  four 
scores  and  accept  that  the  final  benchmark  score  is  870.  The 
critical  question  is  whether  this  procedure  gives  a  result  which 
correctly  balances  the  relative  import  nee  of  the  factors.  This 
question  leads  to  the  possibility  of  deriving  factor  weights  in  a 
more  systematic  way. 

Derivation  of  factor  Weights 

25.  The  proposed  method  of  deriving  weights  uses  a  simple 
version  of  a  "policy  capturing"  model.  A  group  of  judges,  defined 
as  being  "expert"  make  independent  decisions  on  the  relative 
importance  of  factors.  The  policy  of  the  group  is  then  captured 

by  computer  analysis  end  presented  as  an  easily  understood  equation. 
The  best  explanation  of  the  method  is  contained  in  an  allegory 
called  Selecting  a  Harem  -  And  other  Applications  of  the  Policy- 
Capturing  Model  (Christal,,  1968). 

26.  In  the  pilot  study,  a  sample  of  40  category  job  descriptions 
(PTE  equivalent)  was  selected  from  the  data  bank.  Fifteen  judges 

at  the  Major  (E)  and  Lieutenant  Colonel  (E)  level  were  asked  to 
rank  -  order  the  jobs  (from  1  to  40)  on: 

a.  the  four  evaluation  factors,  and 

b.  a  criterion  factor  called  Relative  Worth 

27.  With  the  Relative  Worth  factor,  the  judges  were  asked  to 
consider  the  jobs  in  the  broadest  possible  context  and  to  rank  - 
order  them  on  their  relative  worth  to  the  Australian  Defence  Force. 
The  intention  was  that  the  judges'  deliberations  would  extent 
beyond  the  parochial  interests  of  their  own  Services. 
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28.  The  computer  was  then  given  the  judges*  decisions  on  both 
the  four  evaluation  factors  and  the  criterion  rankings  on  Relative 
Worth.  The  resulting  equation  took  the  following  form. 

Relative  Worth  =  .61  x  Consequences 

+.21  x  Environment 
+.19  x  Difficulty 
+.01  x  Hazard 
-.63  (Constant) 

This  equation  represents  the  considered  opinion  of  the  "Experts" 
from  the  three  Services. 

29.  To  test  the  accuracy  of  the  equation,  a  comparison  was 
made  between  the  judges  rankings  and  the  ranking  derived  from  the 
equation  (Annex  A).  In  regard  to  the  general  accuracy  of  the 
equation,  the  correlation  co-efficient  between  the  two  sets  of 
v-lues  is  .97.  How  this  correlation  may  be  improved  upon  is  sug¬ 
gested  at  Annex  B. 

30.  The  factor  weights  were  derived  from  a  wide  consensus  of 
"expert"  opinion.  The  agreement  between  the  15  judges  was  high. 

TABLE  III  -  CO-EFFICIENTS  OF  AGREEMENT 


Task  Difficulty 

.98 

Consequences  of  Inadequate  Performance 

.97 

Hazard 

.95 

Environment 

.97 

Relative  Worth 

.95 

This  suggests  that  high  confidence  can  be  had  in  decisions  derived  . 
from  these  judgements. 

Calculation  of  Category  Scores 

31.  To  illustrate  the  application  of  the  factor  weights,  the 

hypothetical  case  of  the  Girl  Friday  group  is  carried  to  its  logical 
conclusion. 

Relative  Worth  =  .61  x  250  (Consequences) 

+.21  x  125  (Environment) 

+.19  x  382  (Difficulty) 

+.01  x  150  (Hazard) 

-.63  (Constant)  =  252.2 

This  means  that  the  Girl  Friday  group  falls  on  the  job  evaluation 
scale  at  the  252nd  benchmark. 
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32.  The  calculation  of  the  final  scores  for  about  250  categories 

will  be  done  systematically  by  computer.  Several  small  categories, 
for  which  full  occupational  surveys  could  not  be  justified,  would 
be  assessed  by  a  Joint-Service  work  party. 


Finalization  of  Benchmark  Scale 

33.  In  determining  the  final  placement  of  all  categories  on  a 

benchmark  scale,  the  computer-produced  scores  would  be  reviewed  by 
a  group  of  service  representatives.  In  keeping  with  the  theme  of 
capitalizing  on  the  widest  possible  concensus  of  expert  opinion, 
this  group  might  comprise  as  many  as  thirty  people.  The  rules  for 
decision  making  would  have  to  be  prescribed  vigorously  before 
deliberations  began. 


DISCUSSION 


Relationships  Between  Factors 

34.  The  relationships  between  the  factors  require  comment. 

Does  the  pilot  study  suggest  that  the  factors  are  suitable?  Table 
IV  shows  the  correlation  co-efficients  between  the  pairs  of 
evaluation  factor  scores. 


TABLE  IV  -  EVALUATION  FACTOR  CORRELATIONS 


Task 

Difficulty 

Consequences  of 

Inadequate 

Performance 

Hazard 

UN 

Task 

Difficulty 

Consequences 

of 

Inadequate 

Performance 

.64 

Hazard 

.44 

.70 

Environment 

.49 

.59 

.81 

HO  - 


The  high  correlation  between.  Hazard  and  Environment  confirms  most 
people's  commonsense  notions  about  the  two  phenomena.  The  debat¬ 
able  point  is  whether  both  factors  need  to  be  included  in  the  system. 

35.  Table  V  shows  the  correlation  co-efficients  between  each 

evaluation  factor  and  the  criterion  Relative  Worth. 

TABLE  V  -  CORRELATIONS  -  EVALUATION 
FACTORS  AMSTrSlAIEIvE  WORTH 


Evaluation  Factors 

Relative  Worth 

Task  Difficulty 

.76 

Consequences  of  Inadequate 
Performance 

.94 

Hazard 

.75 

Environment 

.73 

The  very  high  correlation  between  Consequences  of  Inadequate 
Performance  and  Relative  Worth  suggests  that  the  two  factors  are 
thought  of  synonymously  by  military  judges.  Therefore,  in  any 
weighted  equation  on  Relative  Worth,  it  would  be  expected  that 
Consequences  of  Inadequate  Performance  would  be  weighted  heavily. 
This  did,  in  fact,  happen. 


The  Accuracy  of  the  Equation 

36.  The  essential  question  is  whether  the  equation  can  be 
accepted  at  face  value.  Is  it  possible  that  the  co-efficient  of 

.97  is  spuriously  high?  Without  going  into  the  technical  complexities 
of  the  question,  two  validation  exercises  are  suggested. 

37.  The  first  action  is  to  assemble  a  second  independent  group 
of  officers  to  provide  judgements  on  the  Relative  Worth  criterion 
factor.  The  resulting  data  could  then  be  matched  with  the  evalu¬ 
ation  factor  data  provided  by  the  first  group  of  judges.  From 
this  matching  procedure,  a  new  equation  could  be  developed  to 
validate  the  original  equation. 

38.  A  second  possible  action  is  to  apply  the  equation  to  a  set 
of  category  job  descriptions  not  used  in  the  original  pilot  study. 

This  would  confirm  whether  the  equation  is  universally  accurate  in 
covering  all  military  categories. 
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Problem  of  Scaling 

39.  An  outstanding  problem  is  that  of  preparing  reliable  factor 

benchmark  scales  across  all  categories.  The  matter  at  issue  is 
that  a  task  reported  to  be  of  high  relative  difficulty  in  one  cat¬ 
egory  is  deemed  to  be  of  low  relative  difficulty  in  another  category. 
The  inclusion  of  sets  of  common  tasks  in  all  inventories  for  com¬ 
parative  purposes  is  not  as  administratively  easy  as  it  might 
appear.  Expert  opinion  will  be  sought  on  this  matter. 


SUMMARY 


40.  The  thrust  of  this  paper  is  that  the  existing  job  eval¬ 
uation  system  should  be  replaced  by  a  computer  based  system 
tailored  to  the  special  needs  of  the  military  industry. 

41 .  The  criteria  to  be  met  are  as  follows: 

a.  There  must  be  large  representative  sampling  to 
cover  all  job-types  adequately. 

b.  Evaluation  factors  must  provide  reliable 
information. 

c.  Factor  weights  must  be  derived  systematically. 

d.  The  scoring  system  must  be  computerized. 

e.  An  analysis  of  the  judges'  decisions  must  be 
possible  using  standard  statistical  procedures. 

f.  The  costs  must  not  be  prohibitive;  furthermore, 
the  system  must  provide  pay-offs  in  other  areas 
of  management. 

g.  The  Services  must  be  satisfied  that  the  system, 
with  its  military  orientation,  will  properly 
represent  the  unique  contributions  made  by 
sailors,  soldiers  and  airmen  to  the  achievement 
of  Australian  Defence  Force  aims. 

42.  From  the  1978  pilot  study,  the  following  conclusions 
emerge. 

a.  The  evaluation  factors  Task-Difficulty,  Consequences 
of  Inadequate  Performance,  Hazard  and  Environment 
are  such  that  reliable  inform  tion  can  be  gathered 
from  work  supervisors. 

b.  Category  job  descriptions  derived  from  CODAP  data 
provide  adequate  information  for  a  factor  scoring 
exercise. 
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c.  Weights  to  be  applied  to  the  factors  can  be 
determined  by  a  group  of  "expert"  judges. 

d.  The  scoring  is  computerized. 

e.  The  system  criteria  csn  be  met  with  considerably 
less  cost  and  turbulence  than  with  a  traditional 
system.  As  a  bonus,  numerous  other  applications 
are  possible  using  the  same  data. 

43.  In  addition,  the  system  is  sufficiently  flexible  to  permit 
the  addition  of  new  factors  or  the  deletion  of  obsolete  factors. 

44.  There  are  two  matters  requiring  further  investigation. 
First,  there  is  a  need  to  validate  the  original  weighed  equation. 
Secondly,  a  method  will  have  to  be  settled  upon  to  prepare  task 
benchmark  scales  for  each  selected  factor. 

45.  As  a  final  point,  it  is  unlikely  that  Service  Offices 
will  find  every  aspect  of  this  proposal  to  their  taste.  However, 
they  might  contemplate  what  the  future  holds  if  they  persist  with 
an  existing  system  which  is  so  obviously  unsuitable  for  evaluating 
military  "other  ranks"  categories. 
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ANNEX  A 


RELATIVE  WORTH  RAMKIHG5 


Category 


Steward 

Telephone  Operator 
Education  Assistant 
Musician 
Clerk 

General  Clerk 

Clerk  Admin 

Litho  Platemaker 

Motor  Transport  Driver 

Medical  Assistant 

Sheet  Metal  Worker 

Rigger  Parachute 

Driver 

Metalsmith 

Blacksmith 

Cook 

Meet  Electrician 
Fireman 

Motor  Transport  Fitter 
Mechanics  Vehicle 
Marine  Technical  Hull 
Weapons  Mechanic 
Fitter  Electrical 
Electronic  Electrical  Power 
Operator  Radio 
Signalman 

Marine  Technical  Propulsion 
Marine  Technical  Propulsion 
Diesel 

Operator  Keyboard  Cypher 
Fitter  Airframe 
Electrical  Fitter 
Electronic  Technical  Weapons 
Airframe  Fitter 
Fitter  Engine 
Armament  Fitter 
Electronic  Technical 
Communications 
Radio  Technician  Air 
Rifleman 
Engine  Fitter 

Electronic  Technical  Weapons 
Systems 


Judges' 

Rankings 


3.2 
4.0 
5.6 

5.9 

6.9 

7.3 
9.5 

12.5 
12.9 

12.9 

14.3 

15.3 

15.9 

17.6 

17.9 

18.3 

19.1 

19.9 

22.4 

22.9 

23.2 

23.5 

23.7 

24.5 
25.0 
25.2 

25.3 

25.2 

26.3 

27.9 

29.1 

29.7 

29.7 

29.8 

30.5 

30.9 

31.1 

31 .3 

32.0 

32.1 


29 

30 

31 

32.5 

32.5 

34 

35 


Equation 

Rankings 


3.9 

7.4 
3.3 

6.5 
6 . 0 

6.6 
8.1 

1 1  .0 

12.4 

15.4 
17.2 
18.8 

15.1 

17.8 

17.4 

13.8 

19.1 

20.2 
20.1 
21  .4 

22.8 

22.9 

24.1 

26.5 

24.8 

24.5 

26.8 

27.7 

23.3 

30.3 

27.8 

28.4 

29.7 

32.1 

30.2 

28.7 

28.8 
26.9 
32.0 

32.3 
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AMEX  B 


NOTES  ON  IMPROVED  PROCEDURES 


1.  The  pilot  study  shows  that  the  weighting  policy  of  military 

judges  can  be  captured.  Any  replication  of  this  study,  such  as  a 
formal  joint-service  exercise,  would  require  the  same  standards  of 
rigour  if  the  final  result  was  to  be  comparable.  The  following 
discussion  is  directed  primarily  towards  an  enhancement  of  the 
accuracy  of  the  policy  capturing  model. 


Size  of  Job  Descriptions 

2.  In  the  pilot  study,  it  was  administratively  convenient 

to  use  shortened  job  descriptions  on  a  single  sheet  of  paper. 
However,  for  the  official  exercise,  it  might  be  more  satisfying 
for  some  judges  to  have  the  complete  description  available. 


Number  of  Categories  Represented 

3.  Although  most  judges  were  satisfied  with  40  job 
descriptions,  better  results  are  likely  with  a  broader  represent¬ 
ation  of  "military  skills",  aircrew  and  sea-going  categories. 
Currently,  there  is  a  limit  on  the  range  of  categories  available 
from  the  data  bank. 

4.  The  recommendation  that  a  larger  set  of  categories  be 
used  is  based  on  the  well  researched  principle  of  "context  effects". 
This  principle  states  that  more  accurate  rankings  are  obtained 
when  a  job  is  judged  against  numerous  different  types  of  jobs. 

A  set  of  80  -  100  jobs  is  suggested. 


Administration  of  Formal  Weighting  Exercise 

5.  Whereas  the  judges  in  this  study  did  the  ranking  exercise 

at  their  own  convenience,  more  formalized  procedures  would  enhance 
the  reliability  of  the  judgements.  The  procedures  might  include 
the  following: 


a.  The  judges,  assembled  in  a  common  work  area,  would 
receive  identical  sets  of  job  descriptions  and  a 
briefing  on  interpretation  of  occupational  inform¬ 
ation. 

b.  Additional  information,  such  as  the  Manual  of 
Army  Employments,  would  be  available  for  perusal. 

c.  The  judges  would  be  permitted  to  discuss  factual 
matters  with  each  other,  but  the  final  decision 
would  be  their  own. 
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Composition  of  the  Judging  Panel 

6.  The  15  judges  in  the  pilot  study  made  no  claims  to  be 
especially  knowledgeable  about  Australian  Defence  Force  employ¬ 
ments.  However,  the  panel  should  be,  by  definition,  "expert". 
This  means  thst  Service  Office  representatives  should  have  the 
broadest  possible  knowledge  of  other  rank  categories. 

7.  In  regard  to  the  number  of  judges,  Table  I  indicates 
that  30  judges  (3  services  x  10)  would  provide  uniformly  high 
reliability  on  each  factor. 

TABLE  I  -  Agreement  Between  Judges 


Raters 

Calculated 

Estimated 

Factor 

1 

15 

20 

25 

30 

35 

1 .  Job-difficulty 

.742 

.977 

.984 

.989 

.989 

.991 

2  Consequence  of 

Inadequate  Performance 

.656 

.966 

.974 

.979 

.982 

.975 

3  Hazard 

.567 

.952 

.961 

.968 

.974 

.977 

4  Environment 

.647 

.'965 

.974 

.979 

.982 

.985 

5  Relative  Worth 

.947 

.961 

.968 

.974 

.977 

8.  Reported  Levels  of  Confidence.  The  judges  took  an 

average  six  hours  to  complete  the  ranking  exercise.  Thirty-nine 
percent  of  them  said  that  the  job  descriptions  had  too  little 
det:il;  sixty-one  percent  said  that  they  were  "about  right".  In 
regard  to  the  number  of  jobs  in  the  set  (40),  seventeen  percent 
said  that  they  would  have  had  more  confidence  in  their  rankings 
if  the  set  had  been  longer.  When  asked  to  show  on  a  seven-point 
scale  the  level  of  confidence  they  had  with  each  factor,  judges 
responded  as  shown  at  Table  II. 
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TABLE  II  -  Levels  of  Confidence 


Factor 

Mean  (15) 

1  Job  Difficulty 

5.0 

2  Consequences  Of  Inadequate 

4.4 

Performance 

3  Hazard 

4.6 

4  Environment 

5.0 

5  Relative  Worth 

3.0 

These  mean  values  show  that  the  judges,  as  individuals,  had 
morerate  to  low  confidence  in  themselves.  However,  as  shown  in 
Table  I,  the  statistical  reliability  of  the  group  as  a  whole  was 
very  high  on  each  factor.  The  methodological  moral  to  be  drawn 
is  that  the  individual  judge's  reported  feeling  of  confidence 
is  not  a  good  guide  to  practical  group  performance. 


i 
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COMMENTS  BY  SERVICE  REPRESENTATIVES 
ON  THE  PROPOSED  METHOD  OF 
JOB  EVALUATION  FOR  PAY  PURPOSES 


1.  Following  the  presentation  by  WGCDR  Fugill,  representat¬ 

ives  of  the  single-Service  Offices  were  invited  to  make  formal 
comment. 


Commander  W.L.  Taylor,  Directorate  of  Naval  Industrial  Policy 

2.  CMDR  Taylor  opened  the  comment  from  the  Directorate  of 
Naval  Industrial  Policy  by  stating  that  the  proposal  was  only  part 
of  a  far  wider  area  in  the  Defence  Force.  It  could  not  be  the 
complete  solution. 

3.  He  felt  that  much  more  data  was  needed  than  was  evident 
in  the  pilot  study  in  order  to  provide  a  solution  to  the 
unquestionable  weaknesses  of  the  Hay  system.  He  felt  that  the 
proposed  method  seemed  capable  of  taking  a  wider  sample  of  jobs, 
which  was  a  weakness  in  the  Hay  method.  Hopefully,  it  could 
more  fully  reflect  the  unique  aspects  of  jobs  in  a  military 
environment.  However,  the  proposed  method  did  not  provide  the 
total  solution;  it  merely  set  out  to  establish  in-Service  skill 
relativities. 

4.  CMDR  Taylor  felt  that,  having  graded  employments  on  the 
basis  of  agreed  factors,  the  major  task  of  quantifying  the  results 
in  financial  terms  remained.  He  wondered  who  would  translate 
the  results  into  pay  level  terms  and  whether  the  proposed  system 
was  compatible  with  established  industrial  principles. 

5.  CMDR  Taylor  felt  that  the  extent  to  which  the  CODAP 
package  could  accommodate  environmental  employment  changes  was 
not  clear.  He  questioned  whether  any  weighting  process  could 
satisfy  the  variations  in,  say,  hazard  or  job  difficulty  at  sea 
vis-a-vis  the  environment  ashore.  He  asked  how  the  method  could 
accommodate  the  strictly  naval  skills  categories  where  the  true 
skills  can  only  be  demonstrated  at  sea  and  where  employment  ashore 
varied  from  the  primary  duties  at  sea. 

6.  In  summary,  CMDR  Taylor  made  five  points: 

a.  Providing  that  the  Minister  for  Defence  would 
agree  that  the  proposed  method  was  a  suitable 
vehicle  for  the  future,  his  Directorate  saw 
merit  in  further  in-depth  studies  of  the 
proposed  system. 

b.  There  was  a  need  for  some  system  to  improve  on 
the  Hay  System. 

c.  The  need  for  a  more  formalized  methodology 
was  evident. 
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d.  The  need  for  a  contingency  plan  for  the  future 
was  agreed. 

e.  It  must  be  recognized  that  any  such  system  did 
not  provide  the  total  solution.  The  final 
arbiter  had  to  be  external  to  an  in-Service 
benchmark  scale. 


Major  L.R.  Wilson,  Directorate  of  Service  Conditions,  Army  Office 

7.  MAJ  Wilson  stated  that  the  Directorate  of  Service  Conditions, 
Army  Office,  saw  job  evaluation  as  an  aid  -  simply  one  factor  in 

the  pay  determination  process.  Other  factors,  albeit  ones 
difficult  to  quantify,  were  intra  and  inter  Service  relativities, 
community  standards ,  importance  to  the  Service ,  hazard ,  stress , 
and  attraction  and  retention. 

8.  He  saw  job  evaluation  as  attempting  to  measure  clinically 
an  employee's  accomplishment,  or  from  management's  point  of  view 
his  contribution,  by  using  one  factor  common  to  all  employments 
ie.  skill.  To  this  extent  job  evaluation  had  two  advantages: 

a.  By  using  a  factor  common  to  all  employments  it  is 
able  to  relate  one  employment  to  another  in  a 
consistent  standardized  form. 

b.  It  allows  management's  pay  determinations  to 
appear  fair  to  all. 

9.  He  saw  job  evaluation's  disadvantage  in  its  inability  to 
assess  all  the  variables  an  individual  considers  part  of  his 
personal  accomplishment  or  another's  contribution.  In  addition, 
skill  may  not  be  the  dominant  factor  in  assessing  contribution, 
in  spite  of  the  Kerr-Woodward  philosophy  that  rates  of  pay  should 
reflect  the  actual  tasks  carried  out  and  responsibilities  carried. 

10.  He  felt  that  whilst  the  principle  of  'pay  for  what  you 
do1  may  have  application  in  the  trades  area,  care  should  be 
exercised  in  applying  it  in  the  military  skills  area.  In  the 
military  skills  area  it  had  to  be  modified  to  mean  pay  for 
experience,  proficiency  gained  through  training,  and  the  potential 
to  carry  out  operational  functions.  There  was  a  distinct 
possibility  of  underestimating  the  skills  of  the  military  skills 
soldier  as  much  of  that  skill  was  invisible  for  long  periods. 

11.  MAJ  Wilson  stated  that  the  Directorate  of  Service 
Conditions  wanted  a  system  with  the  following  requirements. 

There  should  be  two  scales ,  one  for  trade  skilled  employments  and 
one  for  military  skills.  Employments  on  one  scale  would  not 

be  directly  comparable  with  employments  on  the  other  except, 
perhaps,  at  one  point.  He  foresaw  eventually  a  number  of  scales, 
each  representing  a  cluster  of  like  employments  eg.  driver 
employments,  survey  and  printing  employments.  In  the  technical 
trades  area  these  might  contain  employments  from  the  three 
Services. 
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The  relationship  between  the  clusters  of  employments  should  be 
left  to  industrial  relations  analysts  who  would  take  into  account 
other  factors  such  as  political  and  socio-economic  considerations, 
attraction  and  retention  factors,  and  the  essential  importance 
of  the  employments  in  achieving  the  Army's  role. 

12.  In  summary,  MAJ  Wilson  said  that  he  hoped  to  instil  some 

caution  in  applying  job  evaluation  processes  to  pay  fixation. 


Mr  A.  Chapman,  Directorate  of  Personnel  Services,  Air  Force  Office 

13.  Mr  Chapman  stated  that  many  of  his  points  had  already 

been  covered.  He  raised  three  main  problems,  each  of  a 
technical  nature: 

a.  He  had  doubts  whether  there  would  be  enough 
survey  information  available  to  carry  out  the 
exercise . 

b.  He  felt  that  carrying  out  exercises  on  a  tri- 
Service  basis  would  be  a  real  problem. 

c.  He  could  not  see  how  to  account  for  a  new 
mustering  -  when  a  survey  would  be  carried  out, 
for  example,  to  determine  its  place  on  the  scale. 


14.  In  summary,  Mr  Chapman  felt  that  the  proposal  could  be 

one  input  into  the  system  but  it  should  not  drive  it.  It  could 
perhaps  fill  gaps. 
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PROPOSAL  TO  ESTABLISH  A  BENCHMARK 
SCALE  OF  JOB  WORTH 

Comments  By  Dr  R.E.  Christal 


1  .  Dr  Christal  drew  attention  to  the  similarities  between 

the  proposal  and  the  USAF  Officer  Grade  Requirements  (OGR) 

Study  that  he  had  outlined  previously  in  the  seminar.  He 
felt  that  similar  problems  had  been  encountered.  In  the  OGR, 
the  task  was  to  capture  from  the  minds  of  those  in  the  military 
establishment  their  concept  of  the  meaning  of  "grade”;  in  the 
proposal  under  scrutiny,  the  problem  was  to  capture  from  people 
in  the  Defense  establishment  their  concepts  of  job  worth, 
without  being  initially  concerned  with  what  that  worth  actually 
was.  If  a  Board  could  obtain  good  inter-rater  agreement  on 
the  worth  of  every  job  category,  then  the  basic  problem  was 
solved;  the  Board's  joint  opinion  was  the  answer. 

2.  Factors  and  an  equation  were  needed  for  two  main 
reasons,  which  were  quite  separate  issues; 

a.  When  the  Board  disbanded,  there  was  a  need  to 
have  some  mechanism  to  fit  new  categories  into 
the  scale  in  the  same  way  that  the  Board  would 
have  done. 

b.  There  was  usually  a  need  to  show  that  the 
Board's  evaluations  were  based  on  factors 
that  had  meaning  to  other  people. 

3.  Dr  Christal  then  turned  to  the  specific  proposal 
presented  by  WGCDR  Fugill. 

4.  His  first  concern  was  whether  the  officers  doing  the 
rating  fully  understood  the  jobs  they  were  rating.  He  felt 
that  extreme  effort  should  be  made  to  ensure  that  every  member 
of  the  Board  knew  exactly  the  nature  of  the  jobs  he  was 
rating.  In  this  respect,  he  recommended  against  using 
abbreviated  task  descriptions  because  the  tasks  appearing  at 
the  top  of  a  job  description  are  often  the  housekeeping  tasks; 
the  ones  that  really  separate  jobs  often  appear  towards  the 
middle  of  the  job  description.  He  advised  having  on  call 
experienced  NCOs  from  every  occupation  to  resolve  any  doubts 
on  the  Board  members'  minds.  He  commented  that,  if  this  was 
a  US  exercise,  he  would  feel  more  nervous  if  he  had  officers 
rating  enlisted  jobs  than  if  he  had  officers  rating  officer 
jobs.  He  saw  this  proposal  as  being  particularly  challenging 
because  of  its  tri-service  nature. 

5.  Dr  Christal  criticized  the  method  by  which  the  Board 
rated  according  to  the  factors  and  then  later  according  to 
worth.  He  was  sure  that  induct!  correlation  must  have 
resulted.  The  very  fact  that  the  Board  had  four  factors 
presented  to  them  was  subtle  indication  that  these  were  the 
probable  factors  that  accounted  for  worth.  One  Board 
(called  the  Criterion  Board)  should  concentrate  on  only  one 
problem  ie,  rating  each  category  according  to  their  perception 
of  worth.  Other  sources  should  be  used  to  obtain  the  factor 
information  needed  to  work  up  the  equation  used  to  explain  the 
Criterion  Board's  ratings. 
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6.  Dr  Christal  felt  that  more  factors  should  be  used, 
even  though  some  would  undoubtedly  prove  redundant.  At 
least  their  redundancy  could  then  be  demonstrated.  It  was 
possible  to  establish  a  minimum  subset  of  factors  which  were 
economical  to  deal  with  in  the  ■'’inal  equation. 

7.  Dr  Christal  then  drew  attention  to  the  need  to  ensure 
that  the  equation  was  statistically  stable.  There  needed  to 

be  sufficient  criterion  observations  with  relation  to  the  number 
of  factors  being  rated  to  avoid  the  statistical  problem  of 
" over-f i 1 1 ing" . 

8.  In  his  next  point,  Dr  Christal  stressed  the  need  for  a 
thorough  analysis  of  the  Board's  ratings.  It  was  important 
to  see  not  only  the  overall  inter-rater  agreement  about  jobs, 
but  also  the  agreement  about  each  specific  job.  If  there  was 
violent  disageement  about  a  job,  this  job  could  be  put  aside 
for  the  moment  while  factor  information  was  developed  on  the 
jobs  where  there  was  agreement.  After  the  resultant,  it  could 
then  be  applied  to  the  jobs  where  there  was  disagreement  to 
see  where  those  jobs  actually  fell  in  terms  of  the  criterion. 

In  other  words,  where  there  was  agreement  the  direct  ratings 

of  the  Board  could  be  used,  and  where  there  was  disagreement 
the  equation  could  be  used.  If  there  was  still  disagreement, 
at  least  the  debate  was  narrowed;  it  might  be,  for  example, 
that  the  argument  revolved  around  only  one  factor. 

9.  Further  to  this  point,  Dr  Christal  emphasized  that,  if 
there  was  disagreement,  if  was  important  to  find  out  exactly 
the  sources  of  this  disagreement.  In  order  to  find  out  if 
there  was  differences  in  perspective,  the  raters  might  be  divid¬ 
ed  for  analysis  into  Army,  Navy,  Air  Force,  for  example, 

while  the  jobs  might  be  divided  into  support,  maintenance, 
and  so  on.  It  was  important  to  find  out  how  every  Board  member 
rated  every  job  in  every  category  compared  with  how  every  other 
Board  member  rated  them.  Furthermore,  the  Board  members 
should  know  that  this  would  happen. 

10.  In  summing  up.  Dr  Christal  felt  that  the  major 
contribution  of  CODAP  was  to  provide  good  information  to  the 
Criterion  Board  so  that  every  member  understood  fully  each 
category  he  was  rating.  The  equation  would  be  used  to  help 
resolve  problems  where  there  was  disagreement  and  to  bring  new 
occupations  into  the  system. 
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REVIEW  OF  MILITARY 
JOB-SATISFACTION  RESEARCH 
USING-  OCCUPATIONAL  INFORMATION 

By  Dr  R.E.  Christal 


1 .  The  US  Air  Force  is  interested  in  studies  of  job 
satisfaction  because  of  the  evidence  that  dissatisfied 
workers  tend  to  leave  the  service  at  the  first  opportunity. 

As  you  know  we  no  longer  have  a  military  draft  in  the  United 
States,  and  we  are  experiencing  a  significant  reduction  in  the 
size  of  the  18-21  year  old  population  from  which  we  draw  our 
volunteers.  All  of  the  US  Services  are  experiencing  problems 
in  filling  positions.  There  are  also  problems  associated  with 
high  attrition  rates  and  low  re-engagement  rates  which  make 
force  maintenance  even  more  difficult. 

2.  At  the  present  time,  the  Air  Force  Human  Resources 
Laboratory  (AFHRL)  is  undertaking  research  on  process  models  of 
personnel  turnover.  These  studies  will  involve  continuous 
tracking  of  large  numbers  of  personnel  throughout  their  Air 
Force  careers  to  observe  and  measure  factors  related  to  their 
attitudes  and  career  decisions.  Since  this  research  thrust  is 
just  getting  underway,  I  will  not  describe  it  in  detail  today. 
Rather,  I  will  discuss  a  series  of  past  studies,  most  of  which 
have  made  use  of  occupational  data  and  the  CODAP  analysis 
system. 

3.  Our  past  research  on  job  satisfaction  has  involved  one 
main  long-term  stream,  as  well  as  a  number  of  short  term  studies 
on  specific  problems.  The  long-term  effort  is  broken  into 
five  phases,  as  follows:  (a)  a  review  of  the  state-of-the- 
art;  (b)  definition  of  the  dimensions  of  job  satisfaction; 

(c)  measurement  of  the  satisfaction  of  personnel  in  each 
occupation  on  each  defined  factor;  (d)  establishing  the 
relationship  of  each  factor  to  turnover  in  each  occupation; 
and  (e)  determining  the  changes  in  jobs  and  job  conditions 
which  will  reduce  turnover  in  each  occupation.  A  review  of 
the  job  satisfaction  research  conducted  by  AFHRL  thru  1976  is 
available  (Gould,  1976),  along  with  the  results  of  a  review  of 
400  civilian  and  284  military  publications  (Tuttle  &  Hazel, 
1974). 

4.  Results  of  the  Tuttle  and  Hazel  review  confirmed  that: 
(a)  job  satisfaction  is  related  to  turnover,  absenteeism,  and 
sick  calls;  (b)  the  relationship  between  satisfaction  and 
performance  is  inconclusive;  (c)  job  satisfaction  is  multi¬ 
dimensional;  (d)  no  satisfaction  measurement  instruments  were 
found  which  covered  all  major  aspects  of  military  work  environ¬ 
ments;  (e)  job  satisfaction  is  a  function  of  the  interaction 
between  work  factors  and  personnel  needs;  (f)  very  few  studies 
have  dealt  with  specific  job  content  data  —  utilizing  instead, 
subjective  reports  of  job  attitudes;  (g)  very  few  studies  have 
been  longitudinal;  and  (h)  theories  of  satisfaction  tend  to  be 
oversimplified  with  overlapping,  frequently  conflicting 
research  findings. 

5.  After  a  thorough  review  of  the  literature, 

work  was  undertaken  on  identifying  and  developing  measures 
of  all  of  the  dimensions  of  job  satisfaction  in  the 

military  environment .  An  instrument  called  the 
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Occupational  Attitude  Inventory  (OAI)  was  constructed  and 
administered  to  3000  enlisted  personnel.  The  OAI  contained 
348  attitude  items  which  were  designed  to  cover  the  35 
hypothesized  job  satisfaction  dimensions  listed  in  Column  1 
of  Table  1 .  Data  on  these  items  were  intercorrelated  and 
factor  analyzed.  Thirty-five  factors  were  identified  in  the 
solution  and  are  reported  in  the  second  column  of  Table  1 . 
Based  on  subsequent  analyses,  the  OAI  was  trimmed  down  to  200 
items  and  administered  to  an  additional  10,000  case  sample. 
These  data  are  being  analyzed  at  the  present  time  to  determine 
their  relationship  with  turnover  rates. 


Hypotfl.»u*d  Oim.itflons 

Number  of 
Ordinal 
Item* 

Rotated  Fee  tort 

Achievement 

7 

Additional  Duties 

Activity 

8 

Assignment  Locality 

Ait  Force  and  Unit  Policies  and  Practices 

18 

Base  Housing  and  Eating  Facilities 

Assignment  Locality 

17 

Benefits  Provided  by  Base  Facilities 

Authority 

4 

Control  Over  Others 

Co-workers 

9 

Co-workers 

Creativity 

10 

Economic  Security 

Economic  Security 

4 

Family  Attitude  Toward  Job 

Importance 

8 

Independence 

Independence 

9 

Information  on  Policies  and  Procedures 

Interest 

9 

Job  Security 

Job  Change 

7 

Knowledge  of  Results 

Job  Design 

10 

Leave  and  Time-Off  Policies 

Knowledge  of  Results 

7 

Morality  of  Work 

Optional  Social  Contact 

7 

Opportunities  for  Social  Contact 

Pay  and  Benefits 

12 

Opportunity  to  Stay  Busy 

Performance  Evaluation 

8 

Personal  Growth  and  Development 

Personal  Growth  and  Development 

9 

Physical  Demands  of  Job 

Physical  Safety 

6 

Physical  Safety 

Physical  Work  Environment 

13 

Physical  Work  Environment 

Promotion  Opportunity 

8 

Promotion  Opportunity 

Recognition 

9 

Recognition 

Required  Social  Contact 

10 

Required  Verbal  and  Written  Coordination 

Responsibility 

10 

Service  to  Others 

Service  to  Others 

8 

Social  Contact  (non-peer) 

Social  Status 

11 

Status  in  Civilian  Community 

Sufficiency  of  Training 

12 

Sufficiency  of  Training 

Supervision  Received-Human  Relations 

15 

Supervisor's  Competence 

Supervision  Received-Technical 

9 

TDY  Costs  and  Conditions 

Tools,  Equipment,  and  Supplies 

8 

Tools,  Equipment,  and  Supplies 

Utilization 

8 

Travel  Requirements  and  Opportunities 

Value  of  Experience 

8 

Unit  Safety  and  GMT  Programs 

Variety 

9 

Value  of  Experience 

Work  Schedule 

15 

Work  Itself 

Supervisory  Duties 

18 

Work  Schedule 

Unclassified 

8 

Total 

348 

Table  1 


Hypothesized  35  Job  Satisfaction  Dimensions 
and  35  Empirically  Derived  Varimax  Rotated 
Factors 
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6.  Table  2  presents  example  attitude  items  defining  two 

of  the  35  factors.  A  full  description  of  the  Occupational 
Attitude  Survey  and  of  the  factor  analysis  study  is  given  in 
a  publication  by  Dr.  Bruce  Gould  (Gould,  1978). 

The  Work  Itself 

Challenge  provided  by  your  job 
Amount  of  interesting  work  performed 
Feeling  of  accomplishment 
Importance  of  work 
Use  of  abilities 
Variety  of  tasks 

Supervisor's  Competence 

The  way  your  supervisor  handles  his  subordinates 

The  ability  of  your  supervisor  to  make  decisions 

The  way  your  supervisor  trains  his  men 

The  technical  competence  of  your  supervisor 

Your  supervisor's  knowledge  of  the  way  your  job  is  done 

Table  2.  Example  attitude  items  under  two  job  satisfaction 
factors 


7.  While  we  do  not  have  score  values  for  all  occupations 

on  the  35  OAI  factors,  we  do  have  values  for  most  occupations 
on  three  items  normally  collected  in  the  background  section  of 
job  inventories.  These  are:  Job  Interest;  Utilization  of 
Training  and  Talents;  and  Sense  of  Accomplishment.  Survey 
results  indicate  that  there  is  a  wide  variation  on  levels  of 
satisfaction  on  these  items,  both  within  and  between  occupa¬ 
tions.  For  example,  Table  3  reports  the  percent  of 
individuals  at  the  3,  5  and  7  skill  levels  in  a  number  of 
occupations  who  report  that  their  training  and  talents  are 
being  used  "very  little"  or  "not  at  all".  Notice  the  wide 
differences  between  occupations.  For  example,  at  the 
journeyman  level  (5  skill  level)  54#  of  those  working  in  the 
Pavements  Maintenance  specialty  reported  little  or  no  utiliza¬ 
tion  of  training  and  talents,  while  only  6#  of  those  working 
in  the  Dental  Laboratory  specialty  reported  they  were  being 
poorly  utilized. 


J 
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AFSC 

Career  Ladder 

N 

Skill  Level 

3 

5 

7 

551X0 

Pavements  Maintenance 

1,059 

63 

54 

15 

276X0 

A/c  Control  and  Warning 

1,689 

63 

51 

27 

461X0 

Munitions  Maintenance 

1,590 

58 

50 

21 

402X0 

Aerospace  Photo  Systems 

261 

- 

56 

35 

671X3 

Disbursement  Accounting 

1,354 

58 

45 

NA 

611X0 

Supply  Services 

949 

53 

36 

12 

303X0 

Auto  Tracking  Radar  Repairman 

708 

51 

39 

20 

234X0 

Precision  Photoprocessing 

836 

47 

42 

15 

811X0 

Security 

3,617 

46 

36 

9 

443X0G 

Minuteman  Missile  Mechanic 

1 ,032 

44 

46 

18 

204X0 

Intelligence  Operations 

926 

43 

39 

20 

915X0 

Medical  Materiel 

1,022 

43 

26 

7 

811XQA 

Dog  Handler 

559 

10 

24 

2 

903X0 

Radiology 

548 

9 

12 

11 

432X0 

Jet  Engine  Mechanic 

1,622 

9 

8 

6 

305X3 

Electronics  Computer  Repairman  1 ,055 

7 

14 

13 

607X0 

Loadmaster 

1,773 

6 

12 

5 

324X0 

Precision  Measuring  Equipment 

1,212 

6 

7 

8 

322X1 

Weapons  Control  Systems 

1,285 

5 

6 

3 

272X0 

Air  Traffic  Control 

1,518 

2 

5 

7 

982X0 

Dental  Laboratory 

486 

0 

6 

3 

Table  3.  Rank  Ordering  of  Ladders  by  Percent  Claiming  Talents 

and  Training  are  Utilized  "Very  Little"  or  "Not  at  All" 

8.  It  is  difficult  to  determine  the  relationship  of  job 

satisfaction  factors  with  turnover  in  each  occupation, 
especially  with  cross-sectional  data.  Some  individuals  in 
a  cross-sectional  sample  will  spend  one  to  three  years  in 
service  before  reaching  the  end  of  their  contract.  While 
we  know  their  attitudes  at  one  point  in  time,  we  have  no 
information  on  subsequent  changes  in  their  jobs,  job 
conditions,  or  attitudes.  However,  we  have  developed  a 
method  to  infer  the  probable  power  of  satisfaction  factors  to 
predict  re-engagement  decisions  in  each  occupation  using 
cross-sectional  data.  It  involves  use  of  complex  regression 
analysis  to  predict  factor  levels  as  a  function  of  months  of 
military  service.  The  inference  is  made  possible  by  including 
in  the  analysis  sample  only  those  cases  with  the  same  number 
of  years  of  service  commitment.  Figures  1 ,  2  and  3  present 
plots  of  expected  job  interest  factor  values  as  a  function 
of  months  of  service  for  individuals  in  three  occupational 
categories. 
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MONTHS  IN  AIR  FORCE 


Figure  1 .  Pavements  Maintenance 

9.  Figure  1  presents  data  for  personnel  working  in  the 

Pavements  Maintenance  Occupation.  It  can  be  seen  that  the 
job  interest  of  workers  in  the  first  48  months  of  service 
begins  low,  and  continues  to  drop  throughout  the  period.  At 
this  point,  there  is  a  residualization  process  and  only  those 
who  have  re-engaged  are  included  in  the  sample.  The  data 
indicate  that  those  who  have  re-engaged  are  those  who  find  the 
work  in  this  occupation  interesting,  since  the  regression  line 
shows  a  sharp  jump  at  the  48-month  point.  The  data  also 
indicate  that  those  who  re-engage  find  their  work  in  their 
second  term  to  be  progressively  more  interesting  with  the 
passing  of  time. 


MONTHS  IN  AIR  FORCE 


Figure  2.  Precision  Photoprocess 

10.  Figure  2  presents  data  for  the  Precision  Photo¬ 

processing  occupation.  Individuals  in  this  occupation 
indicate  a  moderate  and  steady  job  interest  level  in  their 
first  enlistment.  However,  there  is  a  very  strong  indication 
that  only  those  who  find  the  work  extremely  interesting  tend 
to  re-engage.  For  those  who  stay  on,  they  find  their  work 
progressively  more  interesting. 
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MONTHS  IN  AIR  FORCE 


Figure  3.  Programmer 

1 1 .  Figure  3  presents  data  for  personnel  in  the  Computer 
Programming  occupation.  The  data  indicate  that  nearly  all 
individuals  in  this  occupation  find  their  work  extremely 
interesting,  although  there  appears  to  be  some  deterioration  in 
interest  during  the  first  48  months.  There  is  no  indication 
that  those  who  re-engaged  find  work  in  this  area  any  more 
interesting  than  those  who  left  the  service. 

12.  While  the  data  presented  in  Figures  1,  2  and  3  are 
based  upon  cross-sectional  data,  a  recent  study  has  been 
conducted  which  suggests  that  the  data  probably  can  be  inter¬ 
preted  as  if  they  were  longitudinal  (Gould,  1976).  This 
being  the  case,  one  might  conclude  that  making  the  job  of 
compute *  programmer  more  interesting  is  not  a  reasonable 
approach  Jo  improving  the  re-engagement  rate.  There  is 
essentially  no  "impact  gap"  on  the  regression  curve  in 
Figure  3  between  first  and  second-term  personnel.  Perhaps  if 
the  curve  for  "perceived  pay  differential  for  military  and 
civil  sector  jobs"  were  plotted  for  this  occupation,  a  large 
impact  gap  would  be  found  at  the  48-month  point,  indicating 
that  an  increase  in  pay  might  be  a  solution  to  high  turnover. 
Figure  2  shows  a  very  large  impact  gap  for  the  Precision 
Photoprocessing  occupation.  The  data  suggest  that  moving  some 
interesting  tasks  performed  in  the  second  term  into  first  term 
jobs  might  reduce  turnover.  Figure  1  suggests  very  serious 
problems  in  the  Pavements  Maintenance  occupation.  Not  only 

is  there  a  large  "impact  gap,"  but  neither  first  nor  second 
term  personnel  find  work  in  this  occupation  very  interesting. 

It  appears  that  this  occupation  would  be  a  good  candidate  for 
intensive  investigation. 

13.  Data  in  Table  3  reveal  that  63$  of  the  apprentices  and 
54$  of  the  journeyman  working  in  the  Pavements  Maintenance 
occupation  report  that  their  talents  and  training  are  being 
utilized  "very  little"  or  "not  at  all.  "  Data  in  Figure  1 
indicate  that  first  term  personnel  in  this  occupation  find 
their  jobs  very  dull.  We  conducted  an  in-depth  analysis  of 
this  specialty  to  see  if  we  could  explain  the  reported 
attitudes.  First,  we  reviewed  the  official  occupational 
description  and  found  that  it  appeared  to  describe  meaningful 
work.  The  mg'or  duties  defined  in  this  description  are  as 
follows: 


a.  Constructs,  maintains,  and  repairs  airfield 
pavement,  surface  mats  and  membranes,  roads, 
streets,  walks,  parking  areas,  and  open 
storage  areas;  aircraft  revetments  and 
associated  draining. 

b.  Inspects,  maintains,  and  repairs  railroad 
beds  and  tracks. 

c.  Operates  concrete  batch,  asphalt  batch,  rock 
crushing,  and  other  fixed  or  mobile  plant 
installations. 

d.  Samples  and  tests  soils,  asphaltic  concrete, 
Portland  cement  concrete,  and  crushed  gravel. 

e.  Performs  other  pavements  maintenance  functions. 

f.  Assists  in  the  use  and  handling  of  explosives. 

g.  Supervises  pavements  maintenance  personnel. 

14.  Next,  we  analyzed  the  entry-level  training  course  and 

found  that  it  covered  210  hours  of  instruction  on  what  appear¬ 
ed  to  be  meaningful  tasks.  The  major  elements  in  the  course 
outline  are  as  follows; 


a. 

Orientation,  soils,  and  railways 

60 

hours 

b. 

Rigid  pavements;  prefabricated  surfaces 

and  shelters 

60 

hours 

c. 

Flexible  pavements  and  vegetation 

control 

60 

hours 

d. 

Soil  chemicals  and  explosives 

30 

hours 

15.  Finally,  we  reviewed  the  job  description  for  first 
term  workers  computed  by  the  CODAP  system,  using  occupation 
survey  data.  Figure  4  lists  the  first  17  tasks  on  this 


description,  which  accounted  for  more  than  one-third  of  all 
worktime.  The  problem  immediately  became  apparent. 


Task 

#  Per 
forming 

* 

Time 

Cum 

1°  Time 

Mow  or  edge  grassed  areas 

64.29 

7.70 

4.95 

Wash  or  clean  equipment 

75.18 

4.97 

8.69 

Control  weed  growth 

56.17 

5.53 

1 1 .80 

Trim  or  remove  trees  or  shrubs 

51.09 

5.25 

14.48 

Service  motorized  equip, 

W/fuel,  oil,  etc 

50.12 

4.28 

16.62 

Plant  trees,  shrubs,  grass  or 
flowers 

45.76 

4.34 

18.61 

Water  or  irrigate  vegetation 

32.69 

5.65 

20.46 

Haul  Loose  const,  mat.,  such 
as  sand  or  gravel 


55.57 


3.13 


22.20 


Task 

%  Per 
forming 

* 

Time 

Cum 
%  Time 

Dump  loose  const,  mat.,  such  as 
sand  or  gravel 

55.81 

3.04 

23.89 

Operate  air  compressors 

57.99 

2.88 

25.56 

Fertilize  vegetation 

36.56 

4.54 

27.22 

Lubricate  operating  equipment 

41.16 

3.74 

28.76 

Dig  trenches  or  ditches  by  hand 

47.70 

3.21 

30.29 

Tighten  loose  bolts  or  attachments' 

42.49 

3.59 

31  .81 

Clean,  lubricate,  or  sharpen  tools 

45.52 

3.26 

33.30 

Sweep  paved  surfaces 

38.50 

3.28 

34.56 

Install  or  repair  fences 

42.74 

2.83 

35.77 

Figure  4.  Job  Description  - 

Pavements 

Maintenance/ 

Equipment  Operator  (55IXO  N-826  First 
Terms) 


Workers  were  spending  very  little  of  their  worktime  on  the 
major  tasks  for  which  they  received  training.  Most  of  their 
worktime  was  being  spent  on  mundane  tasks,  such  as  mowing  grass. 

160  One  important  point  is  that  there  were  a  few 

individuals  who  reported  that  they  found  their  work  interest¬ 
ing  and  their  talents  well  utilized.  We  used  the  CODAP 
system  to  produce  a  job  description  for  these  individuals  and 
found  that  they  were  performing  more  difficult  tasks,  a  greater 
variety  of  tasks,  and  fewer  gardening  tasks.  In  fact,  they 
were  performing  tasks  which  were  very  much  in  line  with  those 
for  which  they  had  received  training  and  which  were  listed  in 
the  official  occupational  description. 

17.  We  went  into  the  field  to  confirm  the  information 

collected  with  the  job  inventory  used  in  the  occupational 
survey  to  see  if  we  could  uncover  reasons  for  the  way 
individuals  were  being  utilized.  We  found  that  the  CODAP  job 
descriptions  were  extremely  accurate.  We  also  were  able  to 
uncover  the  factors  giving  rise  to  the  underutilization  of 
military  personnel.  The  major  problem  was  that  most  meaning¬ 
ful  jobs  were  being  contracted  out  to  firms  in  the  civilian 
sector.  This  was  due  in  part  to  the  unavailability  of 
construction  equipment,  such  as  paving  machines.  We  also 
found  that  side-by-side  civil  service  workers  were  many  times 
being  assigned  the  more  interesting  tasks.  Finally,  we  found 
that  the  cost-accounting  system  discouraged  managers  from 
assigning  new  personnel  to  complicated  tasks,  since  managers 
tended  to  be  rewarded  more  for  the  cost  effectiveness  of  their 
operation  than  for  generating  military  skills. 


18.  At  the  end  of  the  investigation,  we  were  able  to  list 

alternatives  for  management,  based  on  solid  information.  If 
the  Air  Force  did  not  need  military  skills  in  the  Pavement 
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Maintenance  occupation,  then  it  should  completely  eliminate 
this  occupation  from  the  military  structures  and  have  such 
work  performed  by  civil  service  or  civilian  contract  personnel. 
However,  if  military  skill  were  needed,  the  obvious  need  was 
to  contract  out  the  non-military  functions  (such  as  grass 
mowing  and  vegetation  care)  and  assign  the  meaningful  pave¬ 
ments  maintenance  tasks  to  military  personnel.  Three  years 
after  these  recommendations  were  made,  we  resurveyed  the 
Pavements  Maintenance  occupation.  We  were  able  to  demon¬ 
strate  that  military  personnel  were  performing  fewer  grounds 
tasks  and  a  lot  more  work  in  line  with  training  and  official 
job  descriptions.  At  the  same  time,  while  job  satisfaction 
had  fallen  off  in  many  Air  Force  specialties,  the  satisfaction 
of  pavements  maintenance  workers  had  increased  significantly. 

19.  As  mentioned  earlier,  it  is  not  easy  to  determine  the 
relationship  between  job  satisfaction  factors  and  re-engagement 
decisions.  The  regression  model  approach  described  above  may 
help  identify  the  most  likely  factors,  but  a  safer  approach 
would  be  through  long-term  longitudinal  studies.  However, 
even  if  we  have  the  important  factors  in  each  occupation,  a 
more  consequential  question  is  what  can  you  do  to  change  jobs 
and  job  conditions  to  increase  worker  satisfaction.  The 
typical  approach  has  been  job  intervention  studies  in  which 
experts  literally  go  into  organizations  and  modify  jobs  and 
conditions.  Pre-tests  and  post-tests  are  administered  to 
demonstrate  increases  in  job  satisfaction.  However,  studies 
of  this  nature  are  very  expensive  to  conduct  and  may  not 
provide  good  answers  to  the  question.  The  job  redesign 
exercise  upsets  organizations,  and  there  is  a  danger  that 
improvements  in  reported  job  satisfaction  may  simply  be  a 
reflection  of  the  so  called  "Hawthorn  effect"  by  which  indiv¬ 
iduals  are  happier  simply  because  they  are  a  part  of  an 
experiment.  There  is  another  approach  which  I  feel  has  many 
advantages.  It  becomes  feasible  in  the  military  environment 
because  jobs  and  job  conditions  change  as  individuals  are 
reassigned.  It  simply  involves  the  administration  of  job 
inventories  and  job  satisfaction  inventories  to  a  sample  at 

two  points  in  time.  Regression  analysis  is  employed  to 
account  for  changes  in  job  attitudes  as  a  function  of  changes 
in  jobs  and  job  conditions.  This  approach  is  inexpensive, 
does  not  disrupt  operations,  and  is  not  subject  to  the 
Hawthorn  effect. 

20.  While  on  the  surface,  it  would  seem  that  more 
satisfied  workers  would  be  the  most  productive  workers,  there 
is  very  little  evidence  that  such  is  the  case.  Evidently, 
even  many  dissatisfied  workers  continue  to  be  productive 
until  they  have  an  opportunity  to  leave  the  service.  We 
have  a  systematic  research  program  which  is  designed  to 
measure  and  improve  the  productivity  of  individuals  and 
organizations.  However,  even  though  it  relies  heavily  on 
occupational  survey  data  and  CODAP  analyses,  it  is  distinct 
from  our  job  satisfaction  research  program,  so  I  will  not 
describe  it  today. 


4 


21 .  In  summary,  insuring  the  job  satisfaction  of  military 

workers  is  important,  since  dissatisfied  workers  tend  to  leave 
the  service  at  the  first  opportunity .  Data  collected  with 
occupational  surveys  and  analyzed  with  the  CODA?  system  are 
useful  in  identifying  the  sources  of  dissatisfaction  in  the 
job  environment  which  can  be  addressed  by  management. 


I 
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APPLICATION  OF  OCCUPATIONAL  INFORMATION 
TO  JOB  SATISFACTION  STUDIES  IN  THE 
AUSTRALIAN  DEFENCE  FORCE 


Wing  Commander  J.W.K.  Fugill  (Ret) 


INTRODUCTION 


1.  Until  this  decade,  job  satisfaction  studies  in  the 
Australian  Defence  Force  were  severely  limited  by  the  lack  of  a 
comprehensive  occupational  data  bank.  As  this  limitation  no 
longer  applies,  it  is  timely  to  ask  -  what  practical  steps  are 
being  taken  to  use  job  satisfaction  data  for  the  betterment  of 
our  respective  Services?  This  question  is  largely  rhetorical  and 
the  answer  is  embarrassingly  obvious. 

2.  A  second  possible  limitation  is  a  lack  of  knowledge  about 
methods  in  this  extremely  complex  area.  Our  Guest  Speaker  has 
given  us  an  over-view  of  the  range  of  possible  approaches.  How¬ 
ever,  the  aims  of  this  paper  are  modest  by  comparison  but  pertinent 
to  the  Australian  military  scene. 


AIMS 


3.  The  aims  are  to: 

a.  describe  methods  for  identifying  work 
groups  which  report  low  job  satisfaction; 
and 

b.  suggest  ways  in  which  occupational  information 
can  be  applied  to  enhance  job  satisfaction. 

4.  The  general  theme  of  the  paper  is  that  the  prompt 
identification  of  existing  and  potential  job  satisfaction  problems 
will  alert  managers  to  the  need  for  timely  remedial  action.  The 
underlying  premise  is  that,  irrespective  of  whether  re-engagement 
quotas  are  being  filled  now,  the  Services  must  work  for  the  future 
by  building  up  a  reputation  for  being  honest,  concerned  and  active 
in  promoting  the  happiness  of  its  members.  This  does  not  imply 
that  military  workers  have  an  inalienable  right  to  happiness. 

But  it  does  suggest  the  possibility  that  high  job  satisfaction  and 
operational  effectiveness  are  compatible  and  worthwhile  aims. 
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IDENTIFICATION  OF 
LOW  SATISFACTION  WORK  GROUPS 


5.  As  a  starting  point,  job  satisfaction  data  will  be  used 

from  the  Communications  Operatives  study  conducted  in  1975-76. 

In  the  original  report  (November  1977),  the  aim  was  to  propose 
methods  of  interpretation  rather  than  to  write  prescriptions. 


Employment  Categories 

6.  The  twelve  employment  categories  included  Signalman  and 

Radio  Operator  from  the  Navy;  Operator  Keyboard  and  Supervisor 
Communications  Centre  from  the  Army;  and  Telegraphist  and  Telephone 
Operator  from  the  Air  Force.  To  the  extent  possible,  each  category 
was  represented  by  servicemen  and  servicewomen  in  a  variety  of  roles 


Data  Collection 

7.  A  total  of  1500  job  incumbents,  representing  a  75  percent 

sample,  was  asked  to  provide  information  on  tasks  performed  in 
their  current  jobs.  In  addition,  they  gave  information  on  thirty 
background  variables.  Of  particular  importance  to  this  paper  are 
the  variables,  job  interest,  utilization  of  training,  and  re¬ 
engagement  intentions.  Respondents  indicated  on  a  five  point 
relative  scale,  their  attitudes  to  their  current  jobs. 


Data  Analysis 

8.  The  mean  values  for  job  interest  and  utilization  of 
training  were  calculated  for  various  categories,  ranks  and  job- 
types.  These  mean  values  are  called  job  satisfaction  indices 
and,  although  they  must  be  interpreted  with  caution,  they  are 
useful  indicators  of  relative  organisational  health. 

9.  As  a  routine  preliminary  to  the  analysis,  the  basio  job- 
type  diagram  and  associated  job  descriptions  were  prepared. 


Employment  Category  Problems 

10.  Identification  of  Category  Problems.  The  twelve 

employment  categories  were  arranged  from  high  to  low  job  interest 
(Table  1).  The  arrangement  is  based  on  the  responses  from  all 
1 500  incumbents  in  the  sample . 


166  - 


TABLE  1 :  CATEGORY  BY  JOE  INTEREST 


Category 

Position 

Job  Interest 

SPV  COMCEN 

1 

4.36 

OP  KEYBD  &  Cl PH 

2 

3.99 

TELEG 

3 

3.96 

TPRINOPC 

4 

3.88 

RO 

5 

3.87 

TPHONEOP 

6 

3.78 

TELSOPC 

7 

3.72 

OP  KEYBD  &  RAD 

8 

3.69 

OP  RAD 

9 

3.68 

WRRO 

10 

3.63 

SIG 

11 

3.58 

OP  KEYBD 

12 

3.51 

— 

This  Table  shows  that  the  OP  KEYBD  category  reported  the 
lowest  job  interest.  A  study  of  the  job  description  for  that 
category  (Annex  B)  provides  a  means  for  the  manager  to  detect: 

a.  individual  task  performance  which  is  highly 
repetitive ; 

b.  high  incidence  of  task  performance  outside 
the  area  of  specialization;  and 

c.  when  read  in  conjunction  with  the  syllabus, 
low  utilization  of  training. 

Having  identified  the  problem  area,  the  specialist  may  seek  to 
resolve  the  problem. 

11.  Resolution  of  Category  Problems.  Category  problems 

having  a  bearing  on  job  satisfaction  can  be  complex  and  their 
resolution  involves  consideration  of  many  factors.  The  specialist 
officer,  having  considered  the  factors,  may  elect  to: 

a.  implement  more  frequent  job  rotation  at  the 
section  level; 


b.  ensure  that  highly  trained  incumbents  are  not 
employed  out  of  category  for  long  periods  (but 
for  some  low-level  specialties,  out  of  category 
employment  may  be  beneficial) ; 

c.  in  conjunction  with  personnel  staff,  plan 
postings  to  increase  the  variety  of  work 
experience ;  or 

d.  restructure  the  employment  category. 


Ranks  Problems 

12.  Identification  of  Rank  Problems.  As  a  general  rule, 

reported  job  satisfaction  should  rise  with  rank  because  less 
contented  personnel  will  have  withdrawn  themselves  at  re-engagement 
points  or  their  re-engagement  will  not  have  been  recommended.  If 
reported  job  satisfaction  does  not  rise  with  rank,  then  there  is 
something  seriously  wrong  within  the  category.  Table  2  suggests, 
prima  facie,  that  the  situation  in  1975  was  satisfactory.  However, 
specialist  officers  may  decide  that  action  is  required  when  a 
category's  figures  are  significantly  below  the  means  shown  in 
Table  2. 


TABLE  2:  JOE  INTEREST  BY  RANK 


Navy 

Army 

RAAF 

SMN/WR 

3.5 

SIG 

3.5 

AC/ACW 

3.8 

AB/SWR 

3.6 

LCPL 

3.5 

LAC/LACW 

3.7 

LSMN/LWR 

3.8 

CPL 

3.9 

CPL 

3.7 

PO/POWR 

3.8 

SGT 

4.3 

SGT 

4.0 

SSGT 

4.3 

CPO/CPOWR 

4.4 

WO  2 

4.5 

FSGT 

4.1 

WO/WOWR 

4.7 

W01 

4.4 

WOFF 

4.3 

ALL  RANKS 

3.7 

ALL  RANKS 

3.8 

ALL  RANKS 

oo 

13.  Resolution  of  Rank  Problems.  Possible  courses  of  action 

to  solve  rank  problems  include  those  listed  in  paragraph  11.  In 
addition,  the  rank  structure  may  require  re-evaluation  to  ensure 
that  work  performed  is  consistent  with  nominated  status.  This 
means  that  we  seek  to  ensure  that  Sergeants  are  not  doing  the  work 
of  Privates. 


Job  Types  Problems 

14.  Identification  of  Job-Type  Problems.  During  the  routine 

analysis  of  the  1500  jobs,  twenty-four  distinct  job-types  groups 
emerged.  In  Table  3,  the  job-types  are  arranged  in  order  from 
high  job  interest  to  low  job  interest. 


TABLE  5:  JOB- TYPES  ARRANGED 
FROM  HIGH  TO  LOW  JOB  INTEREST 


Position 

GP 

Job 

Interest 

Rank  Equivalents  * 

PTE-LCPL 

CPL 

SGT- 

SSGT 

W02-W01 

HIGH 

1 

283 

4.65 

6 

9 

5 

2 

442 

4.33 

16 

4 

3 

373 

4.32 

8 

1 1 

10 

5 

4 

035 

4.27 

10 

21 

15 

16 

5 

083 

4.26 

1 

10 

22 

5 

6 

598 

4.07 

42 

35 

7 

7 

319 

3.94 

34 

35 

19 

7 

8.5 

538 

3.91 

4 

7 

10 

1 

8.5 

055 

3.91 

1 

9 

13 

10 

229 

3.89 

27 

8 

3 

1 

1 1 

234 

3.83 

3 

6 

8 

7 

12 

595 

3.81 

24 

23 

7 

13 

256 

3.72 

2 

44 

18 

15 

175 

3.62 

97 

35 

3 

15 

006 

3.62 

56 

14 

8 

15 

201 

3.62 

9 

9 

3 

17 

679 

3.60 

9 

36 

17 

3 

18 

138 

3.59 

61 

1 1 

19 

569 

3.58 

32 

8 

20 

392 

3.57 

65 

8 

2 

2 

21 

168 

3.55 

16 

5 

1 

22 

701 

3.53 

22 

7 

1 

23 

059 

3.50 

25 

2 

1 

24 

049 

3.39 

23 

7 

2 

LOW 

*  The  Rank  Equivalents  columns  show  the  distribution 
of  ranks  for  each  job-type. 


-  169  - 


The  job- type  with  the  highest  reported  job  interest  is  No  283. 
This  group  consists  of  LSMN-CPO  concerned  primarily  with  fleet 
communications  and  training.  As  a  general  point,  people  working 
in  training  roles  report  high  job  satisfaction.  On  the  other 
hand,  the  group  with  the  lowest  reported  job  interest  is  No  049« 
This  group  consists  of  PTE-CPL  (E)  from  all  three  Services.  The 
job  description  for  the  group  shows  that  73  percent  of  the 
incumbents  spent  12  percent  of  their  time  on  non-communication 
tasks.  The  possible  significance  of  this  is  discussed  later. 


Resolution  of  Job-Type  Satisfaction  Problems 


15.  Given  that  everything  possible  has  been  done  to  ensure 

frequent  job  rotation  and  to  minimize  out-of-category  employment 
for  highly  trained  specialists,  higher  level  managers  might  use  a 
posting  policy  which  makes  best  use  of  the  high  satisfaction  job- 
types.  This  implies  two  basic  options: 

a.  give  highly  reported  workers  a  succession  of 
high  satisfaction  job-types  to  increase  the 
chances  of  their  retention;  or 


16. 


b.  give  all  workers  alternatively  high  and  low 

satisfaction  job-types  to  ensure  that  everyone 
has  a  "fair  go". 

For  either  option,  posting  staff  would  require: 

a.  a  list  of  major  job-types  showing  their  job 
satisfaction  "status"; 

b.  a  list  of  job-type  locations;  and 

c.  the  numbers  of  workers  of  each  rank  normally 
assigned  to  each  job- type. 


This  information  could  be  represented  by  numerals  and  incorporated 
into  a  position  numbering  system. 


Correlations  Between  Job  Factors 

17.  In  the  Communications  Operatives  study,  the  statistical 

relationships  between  the  factors  were  generally  too  weak  for  any 
practical  purposes.  A  correlation  co-efficient  of  ,42  between  job 
interest  and  utilization  of  training  shows  that  the  incumbents  did 
not  consider  the  two  phenomena  synonymously.  This  means  that  some 
incumbents  saw  no  logical  inconsistency  in  reporting,  on  the  one 
hand,  low  job  interest,  and  on  the  other  hand,  high  utilization  of 
training.  By  comparison,  the  RAAF  Electrical  Fitter  category  had 
a  correlation  of  .80  for  those  factors.  This  indicates  that  they 
tended  to  see  job  interest  and  utilization  of  training  as  being 
largely  synonymous.  In  other  words,  when  utilization  of  training 
is  high,  job  interest  tends  to  be  high.  These  differences  in 
attitude  between  Communications  Operatives  and  Electrical  Fitters 
reflect  their  differing  aptitudes,  levels  of  education,  training 
and  work  experience.  The  methodological  moral  to  be  drawn  is 
that,  in  the  field  of  job  satisfaction,  each  category  must  be 
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considered  as  a  unique  unit.  Therefore,  any  allegations  containing 
such  statements  as:  "Job  Satisfaction  across  the  three  Services  is 

low  - "  can  be  refuted  by  reference  to  a  sample  of  categories  whose 

particulars  are  contained  in  the  occupational  data  bank. 

IS.  On  the  subject  of  employment  on  non-specialist  tasks, 

the  data  on  the  Army  Communications  Operatives  is  revealing.  A 
comparison  was  made  between  reported  utilization  of  training  and 
the  incidence  of  non-specialist  work  at  the  Army  unit  level.  A 
correlation  of  .82  shows  that  as  time  spent  on  non-communications 
tasks  rises,  reported  utilization  of  training  tends  to  fall.  The 
lowest  reported  Army  unit  had  incumbents  spending  13  percent  of 
their  time  on  non-specialist  tasks  and  4  percent  of  their  time  the 
administration  of  stores  and  equipment. 


RE-ENGAGEMENT  INTENTIONS 


19.  Reported  re-engagement  intentions  are  of  interest  to 
managers  for  the  immediate  information  supplied,  but  of  equal 
interest  is  the  possibility  of  predicting  re-engagement  levels. 

In  the  RAAF  Electrical  study  referred  to  earlier,  there  were 
two  useful  predictors  of  re-engagement  intentions.  These  were 
job  interest  (.70),  utilization  of  training  (.66)  and  age  (.61). 

The  commonsense  hypothesis  suggested  by  this  is  that  categories  with 
a  long  history  of  low  job  satisfaction  will  tend  to  have  low  or 
declining  re-engagement  rates. 

20.  One  further  example,  taken  from  the  Army  Vehicle  Trades, 
shows  that  the  relationship  between  utilization  of  training  and 
re-engagement  intentions  was  .82.  This  is  sufficiently  strong  as 
to  make  Utilization  of  Training  a  useful  predictor  for  Vehicle 
Trades  categories. 


SUMMARY 


21 .  The  analysis  of  job  satisfaction  data  shows  that  problem 
areas  can  be  identified  in  terms  of  categories,  rank  levels  and 
job-type  groups.  Once  the  problem  areas  have  been  identified, 
management  can  be  alerted  to  the  need  for  remedial  action. 

22.  This  paper  does  not  suggest  that  job  satisfaction  problems 
should  be  attacked  on  a  broad  front.  Given  the  limitations  on 
resources,  management  should  allocate  resources  to  those  categories, 
ranks  or  job- type  groups  are  in  the  most  need  of  help. 

23.  No  attempt  has  been  made  to  describe  cause  and  effect 
relationships.  However,  the  job  satisfaction  indices  for  job 
interest  and  utilization  of  training  may  be  useful  predictors  of 
re-engagement  intentions.  However,  this  possibility  must  be 
considered  category  by  category  with  no  immediate  promise  of  the 
development  of  universal  laws  of  job  satisfaction. 
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JOB  DIFFICULTY  AMD  APTITUDE  REQUIREMENTS 
MATCHING  THE  MAN  AMD  HIS  JOB 

By  Dr  R.E.  Christal 


1 •  We  presently  are  in  the  latter  stages  of  a  large 

study  in  which  we  are  attempting  to  evaluate  aptitude 
rv-au;  remen ts  for  Air  Force  occupations.  However,  since 
s .  m.-mngers  seem  not  to  recognize  the  benefits  received 
:'r  v.  .-••i-'ction  and  classification  testing,  I  thought  I 
w  ui  :  :  -.-r.i  a  iittie  time  addressing  this  issue  first. 

Kv.i".  *  n,  ,  we  iiave  done  an  inadequate  job  of  merchandising 
relucts.  I  have  heard  of  some  managers  who  went 
i.r  ns  to  suggest  that  we  do  away  with  testing  altogether. 
As  1  review  the  situation,  I  find  that  we  have  defended 
aptitude  tests  on  three  grounds:  (1)  their  ability  to  predict 
performance  on  the  job;  (2)  their  ability  to  predict 
attrition  in  training;  and  (3)  their  ability  to  predict 
course  grades.  Let's  look  at  these  factors,  one  at  a  time. 

2.  First,  let's  consider  job  performance.  Now  let's 

be  honest  about  it.  We  really  don't  have  overpowering 
evidence  that  our  tests  predict  job  performance,  and  informed 
managers  and  operators  know  that  we  don't.  Many  of  these 
individuals  are  of  the  opinion  that  the  key  to  productivity 
is  not  individual  differences  in  aptitude,  but  good  manage¬ 
ment  o  Experience  teaches  them  that  most  of  the  personnel 
they  deal  with  on  a  day-by-day  basis  could  get  the  job  done 
if  they  simply  applied  themselves.  The  individual  differ¬ 
ences  they  observe  are  mostly  motivational,  or  else  are  not 
job  related.  Of  course,  these  managers  are  at  least 
partially  right.  What  they  fail  to  understand  is  that  this 
lack  of  variance  is,  to  a  large  extent,  the  product  of  test¬ 
ing  and  training.  If  managers  in  an  electronics  maintenance 
occupation  were  to  receive  a  random  sample  of  untrained 
personnel  out  of  the  general  population  and  attempted  to 
generate  the  required  skills  on  the  job,  I  can  assure  you 
that  they  would  quickly  become  acutely  aware  of  individual 
differences  in  aptitude.  However,  this  would  not  be  an 
efficient  way  to  run  a  military  service.  We  use  tests  to 
select  and  classify  individuals  into  occupations  such  that 
each  person  has  the  capacity  to  acquire  the  necessary  skills 
for  acceptable  job  performance.  The  training  program,  in 
turn,  is  geared  to  provide  each  trainee  with  these  required 
skills.  If  the  process  is  efficient,  then  there  is  no 
reason  why  tests  should  have  high  validities  for  performance 
on  the  job.  This  is  especially  true  for  occupations 
requiring  simple  application  of  fully-developed  skills. 

The  second  way  we  have  defended  our  tests  is  by 
wing  Mw  well  they  predict  attrition  in  training.  In 
.Mr  Force,  a  washout  in  pilot  training  costs  the  service 
.  ir.:s  of  dollars,  and  the  claim  is  made  that  millions  of 
f  additional  costs  are  avoided  each  year  by  using 
screen  out  applicants  likely  to  fail  in  training, 
face,  this  sounds  like  a  strong  case  for  tests. 

.  wr  that  within  any  training  class,  individuals 
. :  Mtude  scores  wash  out  at  a  much  lower  rate  than 
w.*:'.  Low  scores.  It  is  also  true  that  washouts 
..  . ve .  However,  it  is  not  easy  to  demonstrate 
tests  save  money  by  reducing  washout  rates. 
r.r-  data  extracted  from  the  Army  Air  Forces 
.  ,-y  Program  Research  Report  No  2  (Dubois), 
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Class 

K 

Aptitude 

Percent 

Cutoff 

Eliminees 

44C 

12,232 

3 

15.5 

44F 

9,371 

3 

12.0 

441 

6,466 

4 

19.6 

45A 

6,525 

4 

21  .0 

45D 

1,384 

4 

21.5 

450 

664 

6 

27.4 

Table  1  . 

Attrition  Rates  and  Aptitude  Input 
for  Every  Third  Pilot  Training  Class 
(44C  Thru  450)* 

4.  Table  1  reflects  pass/fail  data  for  every  third  class 
from  44C  through  45G.  In  classes  44C  and  44F,  the  cutting 
score  on  the  aptitude  score  for  entry  was  Stanine  3,  and  the 
average  attrition  rate  was  13.9$.  In  classes  441,  45A,  and 
45D,  the  cutoff  was  raised  to  Stanine  4.  However,  instead  of 
going  down,  the  attrition  rate  increased  to  20.4$.  Finally, 
in  class  45G,  the  cutting  point  was  raised  to  Stanine  6,  yet 
the  attrition  rate  went  up  again  -  clear  up  to  27.4$.  In 
view  of  these  data,  one  might  conclude  that  attrition  in  pilot 
training  would  be  minimized  if  those  cases  having  the  least 
aptitude  were  entered  into  training.  Of  course,  this  is  not 
true.  The  fact  is  that  attrition  rates  were  controlled  by 
administrative  actions,  and  were  not  dependent  on  the  quality 
of  the  input.  The  number  of  pilot  graduates  was  determined 
in  large  part  by  the  number  of  cockpits  to  be  filled.  The 
data  shown  in  Table  1  reflect  actions  taken  toward  the  end  of 
the  war  as  the  number  of  trained  pilots  became  abundant  and 
aircraft  production  was  reduced.  We  have  good  reason  for 
believing  that  the  quality  of  graduates  from  these  classes 
varied,  but  we  cannot  demonstrate  that  the  use  of  tests  saved 
money  by  reducing  attrition  rates.  We  would  have  even  a  more 
difficult  time  demonstrating  the  influence  of  tests  on 
attrition  rates  in  enlisted  courses.  The  number  of  graduates 
from  such  courses  is  ordinarily  programmed  months  in  advance 
to  meet  operational  requirements,  and  fluctuations  in  input 
talent  produce  only  minor  fluctuations  in  attrition  rates. 
During  periods  of  low  quality  input,  it  is  not  uncommon  to 
increase  wash-backs  and  remedial  training  to  maintain 
production  standards. 

5.  A  third  way  we  have  attempted  to  show  the  value  of 
aptitude  tests  is  in  terms  of  their  ability  to  predict  final 
course  grades.  The  fact  that  aptitude  tests  do  predict 
course  grades  is  irrefutable.  There  is  ample  evidence  both 
in  military  technical  reports  and  also  in  the  civilian 
literature.  However,  many  managers  are  not  impressed  by  the 
fact  that  tests  predict  school  grades.  Increases  in  school 
grades  cannot  be  easily  translated  into  precise  statements  of 
benefits. 

6.  In  the  future,  I  feel  we  should  defend  tests  in  terms 
of  their  ability  to  predict  speed  of  skill  acquisition  and 
costs  of  skill  maintenance.  The  US  military  services  spend 
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literally  billions  of  dollars  each  year  supporting  the 
development  and  maintenance  of  skills.  The  more  obvious 
expenditures  are  associated  with  formal  residence  and  on-the- 
job  training  courses,  but  this  is  just  the  top  of  the 
iceberg.  For  example,  the  Air  Force  spends  hundreds  of 
millions  of  dollars  each  year  just  to  maintain  pilot  and 
navigator  skills.  Even  more  costly  is  the  time  individuals 
in  all  services  spend  in  learning  to  perform  new  tasks  as 
they  are  encountered  on  a  day-by-day  and  assignment-by¬ 
assignment  basis.  To  the  extent  that  aptitude  scores  predict 
the  time  required  for  individuals  to  acquire  and  maintain 
skills,  they  can  be  used  to  reduce  costs  and  optimally 
distribute  talent  to  jobs.  There  is  nothing  unique  or  new 
about  the  concept  of  aptitude  scores  predicting  learning 
rate.  For  example,  in  1963,  John  B.  Carroll  recommended 
that  aptitude  be  defined  as  learning  rate  (Carroll,  1963). 

The  first  intelligence  test  developed  by  Alfred  Binet,  back  in 
1904,  was  designed  to  measure  differences  in  the  level  of 
skills  acquired  by  individuals  during  a  constant  time  interval 
(chronological  age).  These  scores  were  later  normed  and 
converted  into  a  score  "mental  age."  A  ratio  of  the  mental 
age  to  chronological  age  was  computed  and  came  to  be  called 
the  Intelligence  Quotient  (IQ).  Regardless  of  the  problems 
associated  with  the  development  and  utilization  of  IQ  scores, 
they  have  been  used  for  years  as  rough  indicators  of  individual 
learning  rates.  In  the  academic  world,  many  tests  are  called 
learning  abilities  measures,  and  have  been  used  for  decades 
by  teachers  to  place  pupils  into  homogeneous  groups  so  as  to 
minimize  variance  in  learning  rates  within  groups.  Tests 
have  been  shown  to  be  valid  predictors  of  school  grades,  both 
in  the  academic  world  of  the  civilian  sector  and  in  all 
military  services,  and  school  grades  can  be  viewed  as  the 
amount  of  content  mastered  by  students  when  learning  time  is 
held  constant.  Aptitude  tests  also  predict  proficiency  test 
scores  in  the  services,  which  are  rough  measures  of  the  amount 
of  content  mastered  by  individuals  at  various  career  points. 

In  Project  UTILITY  (Vineberg  and  Taylor,  1972),  which  was 
conducted  for  the  US  Army  by  the  Human  Resources  Research 
Organization  in  the  late  1960s,  AFQT  scores  were  shown  to  be 
related  to  the  rate  of  skill  acquisition  in  several  occupa¬ 
tional  areas.  Pilot  training  programs  are  generally  locked- 
step.  For  this  reason,  I  have  been  unable  to  locate  data 
demonstrating  that  aptitude  scores  predict  speed  of  skill 
acquisition.  However,  pilot  aptitude  tests  do  predict 
within-class  elimination  for  flying  deficiency,  and  individuals 
in  the  flying  research  area  assure  me  that  slowness  in 
acquiring  skills  is  the  primary  cause  for  such  elimination. 

This  observation  needs  to  be  confirmed  by  carefully  controlled 
research. 

7.  While  the  evidence  that  aptitude  scores  predict 

learning  rate  is  substantial,  most  of  it  is  indirect. 

Controlled  studies  are  needed  before  units  of  aptitude  can  be 
precisely  related  to  units  of  learning  per  unit  time. 
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8.  There  is  even  less  direct  evidence  concerning  the 

relationship  of  level  aptitude  and  the  time  required  for 
skill  maintenance.  However,  it  would  seem  rather  safe  to 
assume  that,  if  aptitude  scores  predict  time  for  acquisition 
of  new  skills,  they  would  also  predict  the  time  required  for 
reacquisition  of  decayed  skills.  A  question  more  open  to 
doubt  is  whether  aptitude  scores  will  predict  rate  of  skill 
decay.  Underwood  (1954)  published  one  summary  paper  in  which 
he  concludes  that,  when  associative  strength  is  held  constant, 
there  are  no  differences  in  forgetting  rates  as  a  function  of 
aptitude  during  the  first  24  hours.  However,  this  study 
dealt  with  laborative  studies  of  associative  learning  and  short 
decay  periods.  The  military  services  should  be  able  to 
provide  more  definitive  answers  concerning  individual  differ¬ 
ences  in  forgetting  rate  as  a  function  of  aptitude. 

9»  One  very  revealing  study  was  reported  by  the  Naval 

Personnel  and  Training  Research  Laboratory  in  1970  (Johnson, 
1970)  which  provided  data  relating  to  the  skill  decay 
question.  The  study  was  based  on  material  being  taught  in 
the  first  phase  of  the  avionics  fundamentals  course. 

Proficiency  was  measured  by  means  of  the  criterion  referenced 
tests  that  had  been  used  to  validate  the  programmed  instruc¬ 
tional  material  used  in  this  phase.  Measures  were  obtained 
on  a  pre-test,  on  an  immediate  post-test,  and  at  intervals 
of  1  day,  7  days,  28  days,  and  96  days  following  the 
original  learning.  It  was  found  that  in  spite  of  a  fairly 
high  level  of  mastery  on  the  immediate  post-tests  and  a 
considerable  amount  of  review,  much  of  the  material  learned 
during  the  first  phase  of  the  course  was  forgotten  by  the 
end  of  the  course.  The  differences  between  individual 
students  were  large  on  the  pre-test,  were  quite  small  on  the 
immediate  post-test,  and  increased  gradually  over  the  remain¬ 
ing  post-tests  until,  by  the  end  of  the  course,  they  were 
almost  as  large  as  they  were  on  the  pre-test.  Although  this 
study  was  based  on  only  a  fairly  small  N,  it  did  provide  a 
set  of  relatively  unique  data.  The  experiment  began  with 
141  students.  Seven  were  dropped  for  administrative 
reasons;  eight  failed  because  of  slow  progress;  21  washed 
back  because  of  slow  progress;  and  17  were  moved  ahead  because 
of  fast  progress.  Thus,  only  85  cases  were  left  in  the  final 
sample,  and  these  cases  were  fairly  homogeneous  in  terms  of 
learning  rate.  In  spite  of  this  homogenization  process, 
data  in  the  study  can  be  reanalyzed  to  reflect  differential 
decay  rates  as  a  function  of  aptitude.  As  can  be  seen  in 
Table  2,  aptitude  scores  account  for  24%  of  the  final  test 
score  variance,  with  original  pool  test  scores  held  constant 
(partial  multiple  R2) .  Although  one  might  argue  that 
associative  strength  was  not  held  constant,  from  a  practical 
standpoint  it  can  be  stated  that  individuals  showed  differ¬ 
ential  decay  rates  in  criterion  referenced  test  scores  as  a 
function  of  their  aptitude  levels. 
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Predictors 


Validities  for 
Final  Post-Test 


Immediate  Post-Test  .185  .430 
Aptitude  Tests  Alone  .312  .559 
Immediate  Post-Test  Plus  Aptitude  .382  .618 
Unique  Contribution  of  Aptitude  Tests  .197  .444 
Aptitude  Tests  with  Immediate  Post-Test 

Scores  Held  Constant  (Partial  and  R)  .242  .492 


Table  2.  Retention  of  Electronics  Fundamentals 
as  a  Function  of  Aptitude 

10.  Aptitude  or  ability  tests  are  used  by  most  military 

services  for  the  selection  and  classification  of  personnel. 

In  the  US  Air  Force,  we  have  established  minimum  score 
standards  for  entry  into  the  service  and  also  minimum  score 
standards  for  entry  into  each  occupation.  The  person-job- 
match  system,  which  I  will  discuss  later,  is  designed  to 
fill  all  vacancies  with  individuals  who  meet  minimum  score 
requirements.  Beyond  this,  it  is  designed  to  offer  the  most 
demanding  occupational  assignments  to  the  most  talented  Air 
Force  applicants.  A  goal  is  optimum  distribution  of 
available  talent.  Before  proceeding,  I  would  like  to  stress 
the  importance  of  efficient  assignment  of  talent.  Very  few 
people  appreciate  the  large  variance  in  learning  requirements 
among  occupations.  In  the  Air  Force,  we  have  enlisted 
occupations  for  which  our  slowest  learners  (those  with  the 
lowest  aptitude)  can  prepare  in  six  weeks.  We  have  other 
occupations  for  which  our  fastest  learners  cannot  be  fully 
trained  in  a  year.  They  are  still  considered  to  be  neophytes 
when  they  reach  their  first  job.  At  the  same  time,  there  are 
large  differences  in  the  learning  abilities  among  individuals. 
Many  articles  in  the  literature  report  ratios  of  1  to  3,  1  to 
7,  1  to  9,  and  even  1  to  60  in  terms  of  ranges  of  learning 
times  of  individuals  to  master  various  types  of  learning 
tasks.  A  recent  reanalysis  of  data  from  project  utility 
brings  this  point  home.  Approximately  400  repairmen  were 
asked  to  perform  a  standardized  list  of  critical  maintenance 
tasks  under  observation.  They  were  scored  by  trained 
observers  on  the  number  of  tasks  performed  correctly. 

Although  the  study  was  based  upon  a  cross-section  sample,  it 
was  possible  to  draw  curves  representing  the  percent  of 
correct  actions  taken  by  workers  as  a  function  of  their 
aptitude  levels  and  months  of  job  experience.  The  results 
were  striking.  Fresh  school  graduates  at  the  75th  centile 
on  the  Mechanical  Aptitude  Index  performed  60  percent  of  the 
tasks  correctly,  while  those  at  the  25th  centile  performed 
only  30  percent  of  the  tasks  correctly.  Thus,  those  at  the 
higher  aptitude  level  performed  twice  as  many  tasks  properly 
as  those  at  the  lower  level.  Of  greater  significance  was 
the  finding  that  it  took  a  full  two  years  of  job  experience 
before  those  at  the  lower  aptitude  level  were  performing 
equal  to  the  beginning  level  for  those  at  the  highest  aptitude 
level.  The  fact  that  these  curves  were  drawn  from  cross- 
sectional  data  make  the  conclusions  conservative,  since  a 
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significant  number  of  low  performers  were  separated  from 
service  during  the  first  two  years. 

11.  Considering  the  wide  range  of  "learning  loads" 
associated  with  occupations  and  the  wide  range  of  learning 
rates  among  individuals,  it  becomes  clear  that  military 
services  should  strive  to  assign  their  fast  learners  to  the 
more  difficult  occupations.  There  is  good  evidence  that  the 
learning  rates  of  individuals  is  measured  fairly  well  by 
aptitude  or  ability  tests.  What  is  missing  in  the  equation 
is  a  dependable  measure  of  the  learning  loads  associated  with 
occupations.  In  the  US  Air  Force,  we  have  established 
aptitude  score  minimums  for  entry  into  each  occupation,  but 
these  values  were  set  by  judgment.  They  were  not  based  on 

a  systematic  analysis  of  job  content.  Before  the  occupation 
survey  and  CODAP  analysis  procedures  were  implemented,  there 
simply  was  not  sufficient  information  concerning  jobs  to 
evaluate  learning  loads.  However,  HQ  USAF  recognized 
several  years  ago  that  the  established  aptitude  minimums 
needed  to  be  validated,  and  they  asked  us  to  use  our  occupa¬ 
tional  data  bank  for  accomplishing  this  goal. 

12.  At  the  onset,  we  recognized  that  it  was  not  possible 
to  defend  any  aptitude  cutting  score  as  a  minimum  below  which 
individuals  could  not  be  trained  successfully  to  perform  in 
an  occupation.  If  an  entry-level  course  were  lengthened, 
then  individuals  with  less  aptitude  could  master  its  content. 
Also,  introduction  of  job  performance  aids  might  enable  lower 
aptitude  personnel  to  perform  successfully  in  an  occupation. 
For  these  and  many  other  reasons,  we  felt  that  we  would  have 
difficulty  in  defending  absolute  aptitude  minimums.  It 
should  be  recognized  that  aptitude  scores  have  no  absolute 
meaning.  They  simply  represent  an  individual's  standing 
relative  to  other  individuals  in  some  standardization 
population.  We  felt  that  on  the  job  side  we  would  be  unable 
to  measure  learning  requirements  in  an  absolute  sense,  but 
that  we  might  be  able  to  measure  the  learning  load  for  any 
occupation  relative  to  other  occupations.  If  one  knows  the 
relative  learning  rates  for  a  sample  of  individuals  and  the 
relative  learning  requirements  for  a  set  of  jobs,  this  is 
sufficient  information  for  making  optimal  classification 
assignments. 

13.  Fortunately,  we  had  been  collecting  task  difficulty 
ratings  from  supervisors  when  we  conducted  our  occupational 
surveys.  Furthermore,  we  had  defined  difficulty  in  terms 
of  "relative  learning  difficulty."  Thus,  we  already  had  on 
hand  sufficient  information  to  evaluate  the  relative  learning 
requirements  for  tasks  within  each  occupation,  and  we  could 
use  the  CODAP  system  to  compute  the  relative  learning  require¬ 
ments  for  tasks  within  each  occupation.  What  we  could  not  do 
was  to  determine  the  difficulty  of  tasks  and  jobs  in  any  one 
occupation  with  those  in  other  occupations.  We  had  to  find 
some  method  for  rescaling  tasks  from  all  occupations  so  that 
they  could  be  compared  directly.  The  method  by  which  this 


was  accomplished  is  too  complex  to  describe  in  this  meeting. 
Let  me  simply  state  that  it  involved  teams  of  contractor 
personnel  who  studied  nearly  10,000  work  tasks  in  the 
operational  environment.  Each  of  these  tasks  was  evaluated 
by  up  to  14  independent  observers  using  a  25-task  benchmark 
scale.  Ratings  were  accepted  only  if  there  was  high  inter¬ 
rater  agreement  among  raters.  Information  concerning  these 
10,000  tasks  were  used  to  rescale  the  remaining  tasks  in  each 
occupation  and,  once  we  had  estimates  of  the  learning  diffic¬ 
ulties  of  all  Air  Force  tasks  on  the  same  scale,  we  were  in  a 
position  to  compute  the  relative  learning  loads  for  all  jobs 
in  those  specialties  on  which  we  had  job  survey  data. 


ASVAB 

Min  AFS 

i 

70 

J 

Procurement 

80 

Accounting 

80 

Disbursement 

80 

Information 

60 

Air  Traffic 

80 

Weather 

60 

Personnel 

60 

Inv  Mgt 

60 

Comm  Center 

50 

Air  Cargo 

50 

Air  Passenger 

40 

Mat  Facilities 

60 

Med  Materiel 

40 

Administration 

(Low) 


(High) 


Bars  =  +  1  Standard  Deviation 

Table  3.  Relative  Difficulty  Levels  for 
1st  Termer  Jobs 

14,  I'm  not  in  a  position  to  present  final  data  coming 

out  of  this  study  since  they  are  being  staffed.  However, 

I  will  show  you  tables  from  an  earlier  probe  study  which 
illustrate  the  types  of  outputs  which  can  be  obtained  using 
the  CODAP  system.  Table  3  presents  information  concerning 
the  present  aptitude  requirements  for  14  occupations  along 
with  the  relative  difficulty  levels  of  first-term  jobs  in 
those  occupations  as  determined  in  the  probe  study.  The 
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vertical  dash  in  the  middle  of  the  bars  represent  the  mean 
difficulty  levels  of  the  first-term  jobs.  A  bar  covers  plus 
one  standard  deviation;  thus  it  represents  the  middle  68 
percent  of  jobs  in  a  specialty.  A  short  bar  indicates  an 
occupation  in  which  jobs  are  homogeneous  in  terms  of 
difficulty.  A  long  bar  indicates  a  wide  variance.  The 
jobs  are  ordered  in  terms  of  relative  difficulty.  It  appears 
from  these  data  that  current  aptitude  requirements  are  out  of 
alignment.  We  Lave  been  selecting  Procurement  Specialists 
at  the  70th  centile;  yet  the  learning  load  (difficulty  level) 
associated  with  this  occupation  is  higher  than  for  jobs  in 
Accounting,  Disbursement,  Information  and  Weather  occupations, 
all  of  which  are  selected  at  the  80th  centile  level.  Medical 
Materiel  has  a  cutting  score  at  the  60th  centile,  but  appears 
to  contain  easier  jobs  than  are  found  in  Air  Passenger  and 
Air  Cargo,  both  of  which  are  selected  at  the  40th  centile. 


General 
itlr 


Accounting  Disbursement 

(A80)  (a80) 


5.0 


Accts  Cont 


4.5 


Travel  Acctg 
Comm  Svcs  Military  Pay 


4.0 


3.5 


Pay  &  Coll 


Inventory 
Management 
(A60  or  G60) 


Ml 

Passenger 


Materiel 

Facilities 


(G4£) 


NORs  Control 

Auth  &  Allow  Reeds  &  Repts 
Item  Research 
Document  Cont 

Passenger 
Traffic  Cont 


Bench 

Stock 

Supply 


Warehouse 


3.0 


2.5 


Aircraft 

Services 


Table  4.  Relative  Aptitude  Requirements  for 
Selected  Job  Types 


15.  Now,  let  us  look  at  variations  in  job  types.  Table 

4  shows  selected  job  types  from  five  of  the  occupations  in 
Table  3.  Notice  that  the  records  and  reports  job  type  in 
the  Air  Passenger  Specialty  (selected  at  the  50th  centile) 
is  more  difficult  than  the  Commercial  Services  job  type  in 
the  Disbursement  Specialty  (selected  at  the  80th  centile). 

At  the  same  time,  the  Aircraft  Services  job  type  in  the  Air 
Passenger  Specialty  is  considerably  less  difficult  than  job 
types  in  the  Materiel  Facilities  Specialty  (selected  at  the 
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40th  centile).  Data  such  as  those  shown  in  Table  4  can  be 
used  in  restructuring  occupations.  In  case  of  talent 
shortages,  some  of  the  low  difficulty  job  types  could  be 
organized  into  new  management  categories  to  be  filled  by 
low-talented  personnel.  Table  5  displays  the  most  time 
consuming  tasks  in  the  Aircraft  Services  job  type. 


Cumulative  sum  of  average  percent  time  spent  by  all  members 

Average  percent  time  spent  by  all  members  . ». 

Average  percent  time  spent  by  members  performing . 

Percent  of  members  performing  . . • 


D-T3K 

Duty/Task  Title 

• 

• 

• 

• 

P  20 

Install  Fleet  Service  Equipment 

• 

87.42 

7.62 

6!  66 

6le6 

P  17 

Deliver  In-flight  Meals  to  Aircraft 

77.36 

7.38 

5.71 

12.37 

G  13 

Load  Fleet  Service  Supplies  in 

Aircraft 

77.99 

7.31 

5.70 

18.08 

P  18 

Pill  Fresh  Water  Tanks  and 

Containers 

81.76 

6.66 

5.45 

23.53 

p  iy 

Plush  and  Fill  Lavatories 

76.73 

6.80 

5.22 

28.75 

G  11 

Inventory  Fleet  Service  Equipment 

Aboard  Acft 

76.10 

6.54 

4.97 

33.72 

C  26 

Inspect  Fleet  Service  Equip 

Installed  in  Acft 

62  26 

6.66 

4.15 

37.87 

K  28 

Load  or  Unload  Passenger  Baggage 

16.98 

20.59 

3.50 

41.36 

P  6 

Clean  Passenger  Aircraft 

57.86 

6.02 

3.48 

44.85 

G  30 

Replenish  Passenger  Service  Kits 

37.11 

7.89 

2.93 

47.77 

H  5 

Clean  Work  Areas 

40.88 

7.07 

2.89 

50.67 

P  25 

Prepare  Fleet  Service  Records  or 

Reports 

53.46 

5.14 

2.75 

53.41 

B  7 

Direct  Fleet  Service  I\incti ons 

44.65 

5.76 

2.57 

55.99 

P  21 

Maintain  Fleet  Service  Records 

Piles 

46.54 

4.67 

2.17 

58.16 

K  15 

Deliver  Passenger  Baggage 

14.47 

14.64 

2.12 

60.27 

G  10 

Install  or  Stow  Fittings  &  Tiedowns 

28.30 

6.91 

1.95 

62.23 

B  45 

Supervise  Installations  of  Fleet 

Svc  Equip  in  Acft 

38.36 

4.88 

1.87 

64.10 

F  5 

Clean  Cargo  Compartments 

36.48 

4.52 

1.65 

65.75 

F  14 

Co-ordinate  Fleet  &  Maint  Operations 

w/Load  Masters  or  Technical 

Representatives 

28.93 

5.43 

1.57 

67.32 

A  11 

Establish  Local  Procedures  for 

Fleet  Service 

24.53 

6.09 

1.49 

63.81 

Table  5.  Aircraft  Services  Job  Cluster 
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TASKS  (N  -  781 )  X 

Perform  Fault  Isolation  of  12A96811  Pentration  Aids  Teat  24.3 

Station 

Perform  Fault  Isolation  of  AN/aUM-11  Head-Up  Display  Test 
Sets  20.3 

Perform  Acceptance  Inspections  of  Installed  Test  Stations 
or  Test  Sets  17.8 

Evaluate  Equipment  Change  or  TO  Change  Proposals  17.4 

Perform  Alignments  or  Adjustments  of  CENPAC  Programmer  Panels  1 6.9 
Purge  or  Change  Cryogenic  Converters  with  Helium  and  Nitrogen  16,2 
Remove  or  Replace  Travelling  Wave  Tubes  15*5 

Remove  or  Replace  Transistors  14*6 

Remove  or  Replace  Relays  13.7 

Remove  or  Install  Test  Stations  or  Test  Sets  12.9 

Remove  or  Replace  Electrical  Motors  12.0 

Remove  or  Replace  Fuses  6.5 


X  ATDFUT  =  14.7,  SDX  =  1.48 
Table  6.  Predicted  Task  Difficulty  Avionics  Areas 


TASKS  (N  +  357)  X 

Troubleshoot  Intrusion  Alarm  Systems  and  Identify  Malfunctions  16.8 
Determine  Power  Circuit  and  Control  Requirements  for  AC 
Equipment  14.9 

Adjust  AC  Controls  or  Sensors  14.0 

Remove  or  Replace  Motor  Starters  or  Controls  13.5 

Install  and  Connect  Interior  Power  Controls  13*0 

Install  Dining  Hall  Appliances  or  Fixtures  12.5 

Repair  Internal  Wiring  or  Connections  12.2 

Troubleshoot  Lighting  Systems  11.6 

Splice  Wiring  or  Cables  11.0 

Inspect  and  Service  Emergency  Battery  Lighting  Units  10.5 

Remove  or  Replace  Fuses  8.5 

Remove  or  Replace  Extension  Cords,  Wiring,  or  Plugs  5*6 


X  ATDPUT  =  10.3,  SDx  =  1.48 
Table  7.  Predicted  Task  Difficulty  Electrician 


16.  Tables  6  and  7  display  selected  tasks  from  two 

specialties  for  which,  individuals  are  selected  using  the 
Electronics  Aptitude  Index.  The  first  is  the  Avionics 
Repairman  Specialty,  and  the  second  is  the  Electrician 
Speciality.  Notice  that  both  specialties  contain  some  easy 
tasks.  However,  the  "average  task  difficulty  of  tasks 
performed  per  unit  time"  (ATDPUT)  by  Avionics  Specialists  is 
more  than  two  standard  deviations  more  difficult  than  those 
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performed  by  Electricians.  In  the  full  CODAP  display,  781 
tasks  are  listed  for  the  Avionics  Specialty.  Review  of 
rescaled  tasks  helps  one  form  a  context  for  interpreting  the 
final  study  results.  We  soon  will  have  rescaled  difficulty 
indexes  for  more  than  100,000  tasks,  and  we  will  have 
computed  rescaled  difficulty  indexes  for  several  hundred 
thousand  Air  Force  jobs.  Studies  of  this  magnitude  would 
be  impossible  without  the  job  survey  and  CODAP  analysis 
system. 

17.  I've  talked  about  the  value  of  aptitude  tests,  and 
attempted  to  show  that  their  primary  utility  lies  in  their 
ability  to  predict  individual  rates  of  skill  acquisition. 

I've  described  a  technology  using  the  occupational  survey 
data  and  the  CODAP  analysis  system  which  enables  us  to 
measure  the  relative  difficulty  levels  of  tasks,  jobs,  job 
types,  and  occupations.  Now,  let's  talk  a  little  about  how 
this  information  can  be  used.  First,  it  can  be  used  to 
bring  aptitude  requirements  for  entry  into  various  occupations 
into  alignment  with  job  difficulties.  In  the  past,  aptitude 
requirements  have  been  established  by  judgment,  and  the 
present  study  indicates  that  in  some  instances  they  are  not 

in  alignment  with  job  demands.  Second,  they  can  be  used 
to  build  contingency  plans  for  manning  the  force  in  the  face 
of  talent  shortages.  Among  the  options  are:  (a)  to  reduce 
requirements  for  the  lowest  difficulty  occupations;  (b)  to 
make  new  management  categories  out  of  low-level  job  types 
and  have  them  manned  with  lower  aptitude  personnel;  and 
(c)  to  engineer  new  jobs  from  simple  tasks  and  have  them 
performed  by  lower  aptitude  personnel.  Third,  the  difficult 
tasks  can  be  studied  for  remedial  actions.  We  are  planning 
to  identify  difficult  tasks  for  which  job  performance  aids 
should  be  produced. 

18.  A  primary  application  of  the  job  difficulty 
information  by  the  US  Air  Force  is  in  its  job  offer  system. 
This  new  computerized  system  weighs  a  number  of  factors  in 
helping  the  Air  Force  to  decide  which  jobs  to  offer  each 
applicant  who  wishes  to  come  into  the  Air  Force.  It 
maximizes  the  probability  of  making  fill.  If  we  are  falling 
behind  in  filling  a  particular  occupation,  the  system 
automatically  adjusts  itself  to  offer  that  occupation  to 
more  qualified  applicants.  It  gives  weight  to  the 
applicant's  job  preferences.  It  services  management  goal's 
such  as  the  equitable  distribution  of  minority _ groups  across 
occupations.  It  forecasts  the  near-term  quality  of  the 
applicant  pool,  so  that  it  will  not  offer  jobs  to  an 
individual  when  better  qualified  applicants  are  expected  in 
the  near  future.  It  gives  weight  to  the  predicted  success 
of  each  individual  in  various  training  courses.  In  the 
near  future,  the  system  will  predict  the  probability  of 
attrition  and  tend  to  offer  high-cost  training  only  to  those 
who  are  likely  to  survive  the  first  enlistment.  We  are 
evaluating  the  strength  and  stamina  requirements  for  occupa¬ 
tions,  so  that  the  system  will  offer  physically  demanding 
jobs  only  to  those  individuals  who  have  the  strength  and 


stamina  to  perform  them.  All  of  these  factors  are  import¬ 
ant.  However,  a  core  feature  of  the  per son- job-match 
algorithm  is  that  it  offers  the  most  difficult  jobs  to  the 
most  tal,ented  applicants.  Until  recently,  we  knew  the 
talent  level  of  each  applicant,  but  we  did  not  have 
accurate  information  about  the  difficulty  of  jobs.  Now, 
thanks  to  our  occupational  data  bank  and  CODAP  analysis 
system,  we  have  good  data  on  the  relative  difficulty  levels 
of  nearly  all  Air  Force  jobs. 
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TRAINING  RESEARCH  IN  THE 
UNITED  STATES 
MILITARY  ESTABLISHMENT 

By  Dr  R.E.  Christal 


1  .  You  have  asked  me  to  give  a  brief  overview  of  training 

research  in  the  United  States  military  establishment.  This 
is  a  large  order  for  one  not  in  the  training  business.  Yc* 
must  understand  that  all  I  can  do  is  to  present  my  own  perspect¬ 
ive,  which  is  like  one  of  the  blind  men  describing  the 
elephant . 

2.  About  the  only  training  research  conducted  in  my  own 
Division  has  been  associated  with  the  establishment  of  training 
requirements.  Although  the  CODAP  system  has  made  a  major 
contribution  to  determining  training  requirements,  there  is  a 
lot  more  to  be  done  before  we  become  really  sophisticated. 

This  is  an  important  research  area  because  you  can  spend 
millions  of  dollars  in  training  on  the  wrong  things. 

3.  The  Air  Force  and  Navy  both  have  supported  research 
on  fully  proceduralized  job  performance  aids.  The  Air  Force 
has  built  aids  for  all  major  tasks  for  an  entire  specialty. 

Tests  indicated  that  individuals  with  very  little  formalized 
training  could  perform  their  jobs  adequately  using  these  aids. 
However,  there  were  negative  reactions  to  this  program.  Many 
individuals  do  not  like  to  have  every  step  of  their  job 
prescribed.  They  want  an  understanding  of  what  they  are 
doing,  and  why  they  are  doing  it.  Another  problem  is  that 

it  is  very  expensive  to  produce  aids  for  all  tasks  in  a  job, 
and  sometimes  the  tasks  change  rapidly  as  components  are  re¬ 
placed.  This  makes  it  expensive  to  keep  aids  current.  It 
looks  as  though,  in  the  future,  we  may  produce  aids  only  for 
those  tasks  which  are  difficult  and  critical.  Individuals 
can  use  the  aids  when  they  feel  a  need  to  do  so.  1  should 
add  that  many  useful  by-products  have  come  from  our  performance 
aids  research.  We  learned  how  to  produce  good  guides  in  an 
efficient  manner.  Some  of  the  technology  is  being  used  to 
prepare  materials  included  in  maintenance  manuals. 

4.  The  US  Navy  still  has  a  health  program  in  the  perform¬ 
ance  aids  area.  Their  approach  is  to  eliminate  some  training 
on  entry-level  courses  and  to  substitute  performance  aids  for 
use  by  individuals  on  the  job. 

5.  The  Air  Force  has  conducted  some  research  on  Computer 
Assisted  Instruction  (CAI)  and  considerable  research  on  Computer 
Managed  Instruction  (CMI).  In  CAI,  the  computer  stores  and 
provides  instructional  materials  to  students  individually  at 
computer  terminals,  but  this  may  be  mixed  with  group  instruction 
on  occasions.  On  the  other  hand,  in  CMI,  the  instructional 
materials  and  the  tests  are  provided  away  from  the  computer 
terminals.  The  computer  is  used  to  score  the  test,  to  grade 
the  students,  and  to  make  certain  prescriptions  about  follow-up 
materials.  There  is  a  third  stream  of  research  concerned 

with  individualized  instruction  which  is  self-paced  and  has  no 
computer  support.  There  are  some  tentative  conclusions  to  be 
drawn  about  these  modes  of  instruction.  From  about  30  studies 
that  I  reviewed,  it  appears  that  CAI  and  CMI  save  about  30 


-  187  - 


percent  of  student  time  in  meeting  standards.  Many  of  the 
studies  have  small  samples,  but  if  you  average  across  them, 
you  find  a  30  percent  saving.  There  seems  to  be  a  slight 
increase,  not  a  decrease,  in  student  attrition.  There  is  a 
consensus  that  student  evaluation  is  vastly  improved.  In 
terms  of  student  attitudes,  nearly  all  studies  showed  that 
the  students  liked  it  better  than  they  liked  group  instruction 
in  the  lock-step  courses.  In  regard  to  the  instructor's 
attitudes,  they  would  rather  be  teaching  in  the  classroom. 

I  have  some  first-hand  experience  of  the  Air  Force's  attempt 
to  implement  CM I  at  Lowry  Air  Force  Base.  The  biggest  problem 
was  with  the  instructors  who  were  brought  up  with  the  old 
system.  It  is  difficult  for  those  who  have  had  the  pleasure 
of  teaching  to  become  only  advisors  when  asked  questions. 

This  is  not  so  rewarding  to  the  instructors.  If  you  are 
going  to  implement  this  technology,  you  may  need  a  new  breed 
of  instructors.  The  question  of  cost  benefits  is  still  being 
debated.  There  is  a  lot  of  information  on  costs  but  not  much 
quantified  information  on  benefits.  Also,  there  have  been 
no  useful  comparisons  between  CAI,  CMI  and  individualized 
instruction  without  computers. 

6.  In  the  Air  Force,  we  have  had  experiments  in  four 
course  areas  using  CMI.  There  are  about  3,000  students  a 
year  going  through  these  courses.  The  system  has  50  student 
terminals,  10  management  terminals,  and  one  central  computer. 

The  research  is  winding  down  now  and  the  next  question  is 
whether  to  implement  the  technology  on  a  wide  scale. 

7.  Without  question,  the  biggest  training  research  thrust 
in  the  United  States  is  associated  with  simulators.  This  work 
involves  many  agencies  with  psychologists  and  engineers  working 
together.  The  US  Army  has  been  very  active  with  first- 
generation  engagement  simulation  systems  (REA1TRAIN) .  In  one 
report,  it  was  claimed  that  individuals  being  trained  in  this 
system  obtained  26  percent  more  distribution  of  opposing  weapons 
while  sustaining  49  percent  fewer  casualties  compared  with 
those  trained  by  conventional  systems.  Also,  the  Army  is 
working  on  training  devices  associated  with  artillery,  air 
defense,  engineering,  infantry,  armor  and  anti-armor.  There 
are  battle  simulation  systems,  marksmanship  systems,  infantry 
remote  target  systems,  and  laser  devices  for  gunnery. 

8.  The  Navy  has  a  large  research  program  in  simulation 
technology.  This  includes  utilization  of  fibre  optics, 
holograph  and  laser  devices.  Other  activities  involve  sim¬ 
ulated  avionic  maintenance  trainers,  a  Navy  aircraft  ordnance 
simulator,  an  aviation  weapon  system  simulator,  weapons 
delivery  simulation,  an  automated  air  intercept  control  trainer, 
a  synthetic  fire  fighter  simulator,  a  submarine  advanced  react¬ 
ive  tactical  training  system,  an  advanced  visual-near  visual 
electro  optic  sensor  simulator,  a  Class  A  electronic  equipment 
maintenance  simulator,  and  a  Navy  war  gaming  system.  Also, 
there  is  a  prototype  combat  system  trainer  designed  for  people 
who  work  on  guided  missile  frigates.  This  is  used  to  train 
people  who  are  in  port  for  long  periods  and  to  enhance  their 
capabilities  with  target  detection  equipment.  Other  programs 
concern  the  Mark  86  Team  Trainer,  the  Terrier  Operator  Trainer, 
the  L  Mark  III  Operational  Team  Trainer,  the  Fire  Control- 
Search  Trainer,  and  the  Mark  92  Operational  Trainer. 
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9.  The  Air  Force  has  done  a  little  work  on  maintenance 
simulators.  They  find  that  bringing  equipment  into  the  class¬ 
room  is  often  expensive.  Also,  it  is  not  very  practicable  for 
training  because  you  cannot  generate  malfunctions  very  easily. 
There  is  a  firm  belief  that  you  can  build  simulators  which  are, 
in  many  instances,  more  practicable  for  training  than  the  real 
equipment.  Of  course,  most  of  the  Air  Force  research  has  to 
do  with  simulation  of  things  associated  with  flying.  We  have 
a  lot  of  basic  research  in  the  visual  area  for  fighter  attack 
simulation.  For  example,  we  have  engineers  and  psychologists 
working  on  higher  resolution  multi-channel  color  projectors 

for  projecting  high  resolution  target  images;  on  methods  to 
enlarge  photographic  images;  and  on  methods  to  decrease  dis¬ 
tortion  in  photographic  windows.  Another  research  area  concerns 
computer  generated  imagery.  At  Williams  Air  Force  Base,  they 
developed  an  inventory  of  anything  you  might  see  from  an 
airplane  over  Arizona.  The  aim  was  to  generate  ail  the 
imagery  on  wraparound  TY  screens,  using  digital  information 
from  computers.  At  first  everything  was  square  but  now  they 
have  cleared  up  the  edges  and  improved  the  imagery.  They 
have  conducted  research  to  determine  the  extent  to  which  motion 
is  essential  in  training.  This  is  very  important  research 
because  if  you  can  do  without  motion,  you  can  avoid  much  of 
the  expense  of  large  simulators.  Present  research  streams 
include  studies  related  to  air-to-ground  delivery  systems, 
air-to-air  combat  simulation,  air  refueling  operation  simulation, 
take-off  and  landing  trainers,  and  tactics  training.  The 
wave  of  the  future  appears  to  be  associated  with  engagement 
simulation. 

10.  To  summarize,  the  big  training  research  push  is  associ¬ 
ated  with  the  use  of  simulation.  Simulation  training  is  often 
cheaper  than  training  on  real  live  equipment.  It  is  safer, 
and  you  can  train  on  some  things  that  are  dangerous  to  train 

on  in  live  equipment.  The  biggest  advantage  and  benefits 
of  simulation  may  not  be  in  saving  money,  but  in  the  fact  that 
it  actually  generates  better  combat  skills.  That  is  a  primary 
goal  of  training.  Cetainly,  you  can  evaluate  performance  a 
lot  better  when  people  are  operating  a  simulator.  The  big 
question  being  addressed  now  is  the  transfer  of  training.  For 
some  operations,  it  has  been  demonstrated  that  people  trained 
in  simulators  perform  better  than  those  trained  in  operational 
aircraft. 

11.  In  the  past,  there  has  been  a  lot  of  similarity  in  the 
research  being  done  across  the  services.  As  a  result,  there 
is  some  "get-your-act-together"  pressure  being  applied.  In 
the  last  couple  of  years,  the  services  have  worked  closer 
together  to  ensure  that  there  is  little  duplication  of  effort. 
There  is  good  transfer  of  information  from  one  service  to 
another.  There  are  committees  which  try  to  identify  the 
research  gaps.  The  inter-service  co-operation  in  trying  to 
solve  common  problems  is  a  very  healthy  movement. 
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MANPOWER  CONTINGENCY  PLANS 
Wing  Commander  J.W.K.  Fugill  (Ret) 
INTRODUCTION 


1 .  Contingency  plans  are  advance  preparations  for  dealing 
with  emergencies  not  considered  in  the  regular  planning  process. 

2.  Philosophy  students  see  contingency  plans  as  hypothetical 
syllogisms  which  take  the  form: 

If  X,  they  Y 

Unfortunately  for  military  planners,  the  beauty  of  this  simple  X 
and  Y  relationship  is  seldom  appreciated  as  they  come  face-to-face 
with  the  realities  of  ill -defined  assumptions,  unknown  factors,  and 
insufficient  and  unreliable  data  about  people's  behaviour. 

3.  The  pre-conditions  for  successful  manpower  contingency 
planning  include: 

a.  the  identification  of  realistic  problems  for  which 
solutions  are  imperative, 

b.  the  willingness  to  persevere  with  long-term  planning 
as  opposed  to  "quick  and  dirty"  solutions, 

c.  the  existence  of  a  comprehensive  data  bank  about 
people's  work  behaviour,  and 

d.  a  set  of  principles  or  guidelines  (models). 


AIM 

4.  The  aims  of  this  paper  are  to: 

a.  pose  several  practical  manpower  problems  which 

may  arise  in  the  event  of  rapid  force  expansion,  and 

b.  demonstrate  how  the  CODAP  job-type  diagram  can  be 
used  to  prepare  contingency  plans  to  meet  such  an 
event . 


JOB-TYPE  DIAGRAMS 


5.  The  preparation  of  the  manpower  contingency  plans  discussed 

in  this  paper  requires  few  resources  additional  to  those  normally 
used  by  the  occupational  analysis  cells.  In  addition  to  the  basic 
job  descriptions  and  summaries  of  variables,  a  Group  Membership 
printout  is  called  for  to  assist  in  the  drawing  of  a  branch  diagram. 
Each  circle  in  the  diagram  represents  a  work  group.  Using 
empirically  determined  guidelines  proposed  by  Archer  in  1966, 
decisions  are  made  as  to  which  groups  represent  significantly 
different  job-types.  The  individuals  within  each  job-type  group 
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perform  the  same  tasks  and  tend  to  spend  about  the  same  time  in 
performing  those  tasks.  The  homogeneity  of  such  groups  offer 
management  safe  options  in  the  development  of  manpower  contingency 
plans . 

6.  Apart  from  the  stated  aims  of  this  paper,  this  paper,  this 

presentation  seeks  to  illustrate  the  versatility  of  the  job-type 
diagram,  which  up  to  now,  has  had  limited  practical  use. 


THE  'LEAST  HAEM1  CONCEPT 


7.  In  the  face  of  a  major  military  threat  to  Australia,  could 
the  Australian  Defence  Force  fill  recruitment  quotas  without  a 
lowering  of  performance  standards?  If  recruits  are  not  available 
with  the  prescribed  mental  and  physical  attributes,  what  reductions 
in  standards  could  be  recommended  which  would  do  the  least  conceiv¬ 
able  harm? 

8.  Job  requirements  standards  were  established,  in  the  first 
instance,  as  a  matter  of  judgement,  largely  subjective.  Adjustments 
made  from  time  to  time  were  based  on  recommendations  by  commanders 
and  trainers  who  were  concerned  with  minimizing  training  failure 
rates,  and  reducing  errors  in  operational  situations.  Obviously, 

an  proposal  to  lower  job  requirement  standards  will  receive  reluctant 
acceptance  by  field  commanders  and  trainers  alike.  However,  in  an 
emergency,  the  extent  to  which  this  reluctance  is  rationally  or 
logically  based  will  be  largely  academic.  In  seeking  to  recommend 
actions  which  will  do  the  least  conceivable  harm,  thought  must  be 
given  to  the  likely  impact  that  such  actions  would  have  on  job 
performance,  career  development,  posting  flexibility,  and  job  sat¬ 
isfaction. 


APTITUDE 


9.  If  there  is  a  diminishing  supply  of  high  aptitude  recruits, 

what  actions  could  be  taken  to  fill  enlistment  quotas  with  least 
jeopardy  to  operational  efficiency? 


low  Difficulty  Job-Types 

10.  The  first  action  is  to  identify  significant  job-types  into 
which  lesser  talented  people  could  be  placed  with  least  harm.  For 
each  job-type  within  a  category,  the  average  task-difficulty  level 
is  calculated  (the  task-difficulty  ratings  are  supplied  by  work 
supervisors).  The  job-types  are  then  arranged  in  order  from  the 
highest  difficulty  to  the  lowest  difficulty.  The  job-type  with 
the  lowest  difficulty  becomes  the  first  candidate  for  aptitude  red¬ 
uction. 

11.  As  a  real-life  example,  the  job-type  diagram  for  Army 
Vehicle  Trades  will  be  used.  There  are  nineteen  distinct  job-types. 

The  job-type  of  lowest  difficulty  is  Stores  Accounting  and  Handling.  * 

There  is  a  small  element  of  minor  adjustments  and  repairs  which 
could  be  catered  for  by  on-the-job  training.  The  use  of  lower 
aptitude  people  in  this  group  would  release  other  members  for  more 
demanding  work. 


-  191 


ARMY  VEHICLE  TRADES 


GROUP 

JOB-TYPE 

DIFFICULTY 

No. 

024 

Instructors 

4.5 

15 

128 

Workshop  Inspection 

4.1 

6 

• 

* 

* 

• 

* 

» 

* 

* 

029 

Training  Administration 

3.8 

23 

082 

Minor  Unit  Repairs 

3.6 

6 

114 

Personnel  Administration 

3.5 

34 

043 

Stores  Handling  and  Accounts  - 
Minor  Repairs 

3.2 

21 

12.  In  practical  terms,  aptitude  requirements  can  be  lowered  on 

an  actuarial  basis  -  for  each  successive  concession,  the  size  of  the 
manpower  pool  becomes  progressively  greater.  As  this  process  rel¬ 
eases  higher  talented  people  for  re-assignment  to  more  difficult 
jobs,  there  is  a  bonus  to  be  gained  in  the  enhancement  of  job  sat¬ 
isfaction. 


Job  Performance  Aids  (JPAs) 

13.  A  second  possible  action  requires  the  identification  of 
sets  of  tasks  which  could  be  performed  by  low  aptitude  people  given 
appropriate  verbal  and  pictorial  job  performance  aids.  The  criteria 
for  the  adoption  of  JPAs  are: 

a.  the  "start"  and  "end"  of  the  task  are  defined  clearly, 

b.  task  procedures  have  only  minor  variations, 

c.  decision  options  are  clear,  and 

d.  the  instructions  can  be  proceduralized  through 
simplified  expression,  pictures  and  diagrams. 

14.  The  advantages  of  JPAs  are  the  reductions  in  training  time 
and  performance  time  once  habit  patterns  have  been  established.  The 
early  productivity  engendered  by  the  use  of  JPAs  will  enhance  the 
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job  satisfaction  of  those  individuals  whose  ability  is  the  most 
limited.  On  the  negative  side,  job  incumbents  who  have  been  trained 
to  rely  on  JPAs  may  not  perform  satisfactorily  on  non-routine  tasks. 
They  will  not  become  part  of  the  senior  supervisory  pool  whose 
members  can  solve  difficult  and  unique  problems. 


EDUCATION 


15.  If  there  is  a  diminishing  supply  of  educationally  qualified 

recruits,  what  actions  could  be  taken  to  fill  enlistment  quotas 
with  least  jeopardy  to  operational  efficiency? 


Low  Education  Job-Types 

16.  This  method  requires  identification  of  job-types  for  which 

normal  education  requirements  may  be  waived  with  least  detriment  to 
work  performance.  Work  supervisors  would  be  asked  to  provide 
ratings  on  the  levels  of  reading  comprehension  or  mathematics 
inherent  in  each  task.  Job-types  requiring  least  reading  and/or 
mathematical  ability  would  be  the  first  candidates  for  induction 
of  the  lesser  education.  This  action  would  release  the  better 
educated  individuals  for  higher  knowledge-oriented  jobs. 


Creation  of  New  Job-Types 

17.  A  second  approach  is  to  identify  easy  tasks  which  might  be 

separated  out  of  existing  jobs  and  engineered  into  new  jobs  for 
lesser  qualified  people.  The  criteria  for  establishing  new  jobs 
is  that  they  be  meaningful,  full-time,  and  not  dispersed  across 
widely  separated  locations.  An  example  of  a  re-engineered  job 
would  be  the  aircraft  ground  handler.  The  ground  handling  tasks 
would  be  taken  over  by  this  individual,  thus  releasing  aircraft 
tradesmen  for  more  difficult  work.  Training  courses  for  handlers 
would  have  limited  objectives  and,  therefore,  training  time  would 
be  minimal.  As  a  caveat  to  this  proposal,  it  is  noteworthy  that 
in  1973  RAAF  Engine  Fitters  and  Airframe  Fitters  in  their  first 
year  of  work  spent  about  1 1  percent  of  their  time  on  aircraft  hand¬ 
ling  and  cleaning. 


Reading  Comprehension  Levels 

18.  A  third  possibility  is  a  general  review  of  the  reading 

comprehension  levels  of  technical  and  administrative  orders.  A 
study  reported  from  Nellis  Air  Force  Base  in  1971  showed  that  some 
aircraft  maintenance  documents  required  reading  ability  at  second 
year  university  level.  However,  an  especially  pertinent  finding  was 
that  the  incidence  of  non-compliance  to  maintenance  orders  correlated 
highly  with  the  reading  levels  of  airmen  committing  offences.  Airmen 
of  low  reading  ability  were  much  more  prone  to  errors  than  airmen  of 
high  reading  ability.  Since  out  Services  are  operating  equipment  of 
comparable  complexity  to  the  US  Services,  a  review  of  reading  mater¬ 
ials  might  be  incorporated  into  any  forward-looking  plans. 
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TRAINING 


19.  If  pressure  is  applied  for  a  general  reduction  in  training 

time,  what  actions  could  be  taken  to  meet  such  a  demand  with  least 
jeopardy  to  operational  efficiency? 


Job-Type  Training 

20.  The  underlying  concept  is  work-oriented  job-type  training 

for  some  as  opposed  to  broad  category  training  for  all.  Obvious 
examples  are  aircraft  handlers,  battery  shop  processors  and  storemen 
in  technical  categories.  If  real  savings  can  be  made  with  job-type 
training,  then  it  can  be  argued  that  traditional  category  training 
could  continue  without  reductions  in  time.  The  job-type  training 
concept  is  illustrated  by  reference  to  the  MAP  Electrical  Fitter 
diagram.  The  Battery  Shop  group  perform  tasks  which  require 
considerable  less  training  for  satisfactory  performance. 


ELECTRICAL  FITTER  N  =  528 


The  Battery  Shop  attendants  represent  three  percent  of  the  category  - 
about  27  people.  If  the  category  had  to  expand  five-fold,  then  the 
job-type  training  goal  is  135.  With  the  job-type  job  description, 
an  estimate  can  be  made  of  the  number  of  training  weeks  required  to 
fulfill  that  goal.  For  the  Ground  Servicing  Equipment  group,  main- 


* 
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tenance  personnel  could  be  recruited  directly  from  civilian  life 
because  the  work  required  is  similar  to  that  performed  by  auto¬ 
motive  electricians. 


Category  Training  Reductions 

21 .  Having  suggested  the  first  sequential  action  to  be  taken, 

the  second  possibility  is  to  make  reductions  in  category  training. 
Occupational  survey  information  can  identify  areas  of  irrelevant 
training  or  areas  of  lesser  relevance.  Task  can  be  rank-ordered 
on  the  basis  of  relevance.  The  tasks  of  least  relevance  can  be 
omitted  first,  and  as  pressure  for  reductions  in  training  time 
increases,  more  tasks  can  be  eliminated  progressively.  Captain 
Gorman's  paper  on  the  Electronics  Principles  Inventory  (EPI)  is 
especially  germane.  The  general  conclusion  was  that,  across  nine 
engineering  categories,  there  was  an  insufficient  number  of  prin¬ 
ciples  employed  to  justify  a  common  course  in  electronics  training. 
By  the  elimination  of  irrelevant  electronics  training,  a  worker  can 
become  productive  earlier,  and  for  those  talented  people  who  are 
ear-marked  for  problem  solving  at  the  supervisor  level,  short  but 
highly  relevant  course  could  be  provided  if  and  when  required. 


PHYSICAL  REQUIREMENTS 

22.  If  there  is  a  diminishing  supply  of  physically  qualified 
recruits,  what  actions  could  be  taken  to  fill  enlistment  quotas  with 
least  jeopardy  to  operational  efficiency? 

23.  The  method  requires  the  identification  of  job-types  which 
could  admit  females  and/or  males  at  lower  physical  standards  than 
those  currently  prescribed.  This  action  releases  better  qualified 
people  for  more  physically  rigorous  work.  The  lowering  of  physical 
standards  might  be  considered  hand-in-hand  with  the  lowering  of 
performance  standards  in  recruit  training.  As  with  the  previous 
plans,  the  basic  task  data  is  supplied  by  work  supervisors  at  the 
coal  face  rather  then  by  self-appointed  "experts"  who  may  be  remote 
from  the  realities  of  physical  labour  in  the  operational  field. 


SUPERVISION 

24.  Given  the  unwanted  situation  of  accepting  personnel  of 
lesser  aptitude,  education  and  experience,  what  impact  would  this 
have  on  supervisory  standards? 

25.  This  proposal  does  not  entertain  the  idea  of  lowering  job 
requirement  standards  for  supervisors.  Obviously,  with  a  substan¬ 
tial  intake  of  lesser  qualified  people,  the  supervisors  job  becomes 
proportionally  more  difficult. 

26.  The  problem  is  to  develop  a  system  of  career  progression 
which  will  best  prepare  supervisors  for  increased  supervisory 
responsibility.  The  method  involves  the  drawing  of  a  career  path 
through  the  widest  possible  range  of  job-types  with  emphasis  on 
supervisory  training  and  experience.  In  peacetime,  an  individual’s 
experience  is  often  optimized  by  placing  him  in  the  same  job-type 
during  several  successive  postings.  Although  this  makes  the  worker 
immediately  useful  and  minimizes  the  need  for  on-the-job  training,  it 
is  detrimental  to  individual  career  development  and  against  the  long- 
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term  interests  of  the  Service.  One  solution  is  to  adopt  a  deliber¬ 
ate  mis-matching  policy.  As  an  example,  an  encumbent  who  has  a  few 
easy  tasks  in  his  present  job,  would  be  posted  to  a  job-type  invol¬ 
ving  numerous  tasks,  including  difficult  and  unusual  tasks.  In 
the  RAAF  Clerk  Supply  Mustering  (June  1973)*  a  Forward  Supply  Clerk 
performed  an  average  of  five  tasks. 


CLERK  SUPPLY  N  =  513 


The  job- type  diagram  shows  that  there  are  18  other  job-types,  at  the 
3ame  rank,  which  would  provide  wider  experience.  The  job  descrip¬ 
tions  for  each  of  those  job-types  lists  all  the  tasks  that  incumbents 
would  be  required  to  perform.  A  second  example  concerns  senior 
supervisors.  In  Group  108,  HQSC  Item  Supervisors,  the  incumbents 
perform  an  average  56  tasks  and  have  one  subordinate.  A  posting 
to  Group  094,  Supervisor,  would  require  the  incumbent  to  perform  an 
average  of  150  tasks  and  supervise  8  subordinates.  Group  094  is  a 
cluster  of  160  individuals  distributed  into  four  distinct  job- types. 

27.  This  job- type  mis-matching  model  can  be  illustrated  further 

by  a  set  of  options: 

a.  progress  from  a  few  tasks  to  numerous  tasks, 
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b.  progress  from  an  easy  job-type  to  a  difficult 
job-type, 

c.  progress  to  an  easier  job-type  if  evaluation 
reports  indicate  such  a  need, 

d.  progress  from  a  few  subordinates  to  many 
subordinates,  or 

e.  progress  from  low  satisfaction  to  high 
satisfaction  job-type. 

28.  Finally,  these  concepts  can  be  meshed  together  to  form  a 
career  progression  model.  This  model  defines  the  job-type  path 
to  be  followed  if  an  incumbent  is  to  gain  the  widest  possible  exp¬ 
erience.  The  model  is  drawn  to  serve  the  Service's  best  interests 
and  may  not  co-incide  with  the  individual's  personal  choices. 

29.  Although  the  contingency  plans  concerning  aptitude,  educa¬ 
tion,  training  and  physical  requirements  may  never  require  imple¬ 
mentation,  the  plans  for  the  "ideal"  career  progression  to  enhance 
experience  requires  peace-time  application.  In  this  sense,  it  is 
not  a  contingency  plan,  but  a  pre-condition  for  the  implementation 
of  the  contingency  plans. 


STOMARY 

30.  Possible  sequential  actionsto  be  followed  are  summarized 

as  follows: 


APTITUDE 

EDUCATION 

TRAINING 

PHYSICAL 

SUPERVISION 


low  difficulty  job-types 

job  performance  aids 

low  education  job-types 

create  new  job-types 

reduce  document  reading  levels 

job-type  training 

reduced  category  training 

lower  male  standards 

increase  female  work  scope 

mis-matching  policy  NOW 

"ideal"  career  progression  NOW 


31 .  Finally,  the  practical  people  will  want  to  know  what  res¬ 

ources  are  required  for  this  sort  of  planning.  If  you  are  a 
specialist  with  the  planning  responsibilities  for  three  categories, 
then  you  will  need  three  job-type  diagrams  and  associated  job- 
descriptions.  Usually,  these  printouts  will  be  produced  routinely 
by  the  analysis  cells.  But,  in  addition,  the  specialist  may  ask 
for  task  data  on  such  factors  as  difficulty,  reading  comprehension 
levels,  physical  requirements,  and  electronic  knowledge  content. 
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32.  As  stated  at  the  outset,  this  proposal  has  no  future  -unless 

certain  conditions  prevail.  Managers  have  to  recognize  that  the 
contingency  plan  problem  is  worth  solving,  and  they  must  persevere 
with  long-term  planning  as  opposed  to  "quick  and  dirty"  solutions. 
The  systematic  methods  proposed  at  this  seminar  can  do  much  to  make 
planning  a  "low  risk"  enterprise. 


****** 
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CLOSING  ADDRESS 


Major  General  J.C.  Hughes.  DSO,  MC 


Ladies  and  Gentlemen, 

1 .  We  have  come  to  the  conclusion  of  the  Seminar.  Before 
closing  it,  however  I  would  like  to  spend  a  few  minutes  summarising 
the  main  points  that  have  emerged  over  the  last  four  days. 

2.  First .  the  advantage  of  Data  Collection  and  Analysis 
using  CODAP  have  become  clear: 

a.  the  information  is  comprehensive  and  is 
provided  by  on-the-job  experts, 

b.  the  information  comes  from  a  high  percentage 
of  the  job  incumbents, 

c.  the  data  is  reliable,  and 

d.  the  analysis  of  the  data  can  be  presented 
in  a  great  variety  of  formats. 

3.  Second,  CODAP  has  uses  in  a  wide  variety  of  areas: 

a.  Military  training, 

b.  Trade  testing, 

c.  Rank  determination, 

d.  Job  evaluation, 

e.  Job  satisfaction  studies, 

f.  Matching  the  man  and  the  job,  and 

g.  Manpower  planning, 

to  mention  a  few  that  have  been  discussed  during  this  seminar. 

4.  Third,  we  have  been  shown  a  way  ahead.  Speaking  for 
my  own  division,  it  is  quite  clear  from  the  last  session  on 
Tuesday  afternoon  that  my  Special  Studies  Section  has  an  important 
role  to  play  in  the  coordination  of  occupational  analysis 
activities  of  the  three  Services.  It  is  also  quite  clear  that 
the  proposed  method  of  Job  Evaluation  for  "Other  Ranks"  jobs 
needs  some  more  homework.  We  will  consult  with  Dr  Christal 

next  week  and  the  proposal  will  be  presented  again. 


5.  I  wish  to  thank  all  those  who  have  presented  papers  and 
contributed  to  the  discussions.  Without  their  efforts  we  would 
not  have  had  a  Seminar. 
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6.  Our  indebtedness  to  our  guest  speaker  cannot  be  stated 
in  just  a  few  words.  He  has  done  everything  possible  to  set 
us  on  the  right  course.  It  remains  for  us  to  make  an  intelligent 
assessment  of  our  strengths  and  weaknesses  and  to  plan  carefully 
for  the  future. 

7.  Prom  discussions  with  Dr  Christal's  pupils,  I  know  that 
his  purpose  in  life  has  been  to  increase  the  sum  of  human 
happiness.  This  aim  is  completely  consistent  with  his  private 
life  as  a  family  man  and  church  leader.  We  can  judge  ourselves 
to  be  very  fortunate  that  he  chose  the  Military  sphere  in  which  to 
carry  out  his  life’s  work. 

8.  Other  countries,  such  as  the  UK,  Canada  and  West  Germany 
have  all  benefitted  from  Dr  Christal's  work,  but  we  have  received 
very  special  treatment.  I  hope  that  we  can  now  do  justice  to 
this  special,  treatment. 

9.  Would  you  all  join  with  me  in  showing  our  appreciation 
for  his  efforts  on  our  behalf  over  the  last  few  days.  Dr  Christal. 

10.  Thank  you  Ladies  and  Gentlemen. 


Primed  by  Cl.  TMompion,  Commonwealth  Government  Printer,  Canberra 


